That the Massachusetts Senate race is now a toss up, with some polls now putting Republican Scott Brown in the lead, is rocking the American political landscape. After all, the campaign to replace Ted Kennedy takes place in a state Barack Obama carried by 26% in 2008, one which Gallup last August showed leaning Democratic by a whopping 34-point margin.
But Brown's surge past the flat-lining Martha Coakley isn't merely the result of a dismal economy and the tea party backlash against President Obama. From 1990 until 2006, the Bay State elected Republican governors. Changing demographics, feeble Democratic turnout during non-presidential election years and the rise of independents as the largest voting block go a long way in explaining why.
In 2003, former Clinton-Gore adviser Elaine Kamarck looked at the commonwealth's conundrum in an article (registration required) titled, "Glass Ceiling: Why Democrats Can't Elect a Governor." Kamarck pondered why the overwhelmingly Democratic states of Massachusetts and New York persisted in electing Republican governors throughout the 90's and early 00's. Her research suggested that a major change in voter turnout patterns from less well-off, less educated residents to more affluent and better schooled suburban dwellers. Issues like an increase in the minimum wage, a change supported by all five Massachusetts Democratic primary candidates in 2002, affected less than 5% of actual voters.
In March 2005, the Massachusetts Democratic Party published a post-mortem study on its two decade losing streak. "Recapturing the Corner Office" was the result of statewide outreach effort to understand the Party's failure:

In the fall of 2002 yet another loss at the ballot box left many Democratic Party faithful scratching their heads in disbelief. Weld, Weld, Cellucci, Romney; 2002 marked the fourth consecutive election where a Republican candidate had defeated the Democrat in the race for Governor of Massachusetts. In a state where Democrats hold hefty margins in both the House and Senate, the message from voters could not be ignored.

As the authors suggested, the image of the union-dominated, Democratic machine in Boston controlling Massachusetts politics was a thing of the past.
After Mitt Romney beat State Treasurer Shannon O'Brien by 50% to 45% in the 2002 gubernatorial race, pollster Celinda Lake called turnout "the single most influential factor." "Recapturing the Corner Office" summed up the problem, noting the substantial gap in urban-suburban turnout:

In 2002, turnout in solidly Democratic Greater Boston lagged behind suburban communities where O'Brien lost. A problem for O'Brien was the percentage turnout was only 48.6% in the 28 largest (mostly urban) communities that she won, while it was 60.7% in the 322 other communities that she lost. In this environment, a key to winning is persuading and turning out largely independent suburban voters to vote for the Democrats.
Significant correlation and a high ratio of votes for Republicans to the percentage of unenrolled voters does give credence to the declaration that the hi-tech corridor (cities with the most unenrolled voters) is the state's political "center of gravity". Democratic field efforts should, accordingly, reflect this shift.

And for several years, those" unenrolled" voters have been the largest block in the state. When Kamarck examined party affiliation in 2003, Democrats outnumbered Republicans by 36% to 13%, but independents constituted almost half of Massachusetts voters. While Democrats produced a heavy turnout for Barack Obama in 2008 (43% of the vote, compared to 40% for independents and 17% for Republicans), in 2010 the Senate race will likely see Massachusetts return to form.
FiveThirtyEight's Nate Silver laid out the numbers ten days ago. While disputing some of the methodology of a Rasmussen poll showing the hardliner Scott Brown making great gains in the polls, Silver made clear the election would be won or lost among the independents who now dominate Massachusetts politics in the Route 128 corridor and beyond:

Although there are lots of different ways to ask about party identification, typically that's not what we see in elections in the Bay State, as the number of independents is usually much higher (43 percent of Massachusetts voters were independent/other in 2008, and 51 percent are registered as independents). They're also showing an electorate that is 39 percent liberal, 34 percent conservative, and 27 percent moderate; that compares to 2008 exit poll demographics of 31 percent liberal, 19 percent conservative, and 49 percent moderate.

To be sure, the 2010 Massachusetts Senate race is a unique one. For the first time since 1962, Ted Kennedy is not on the ballot. The brutal recession and the heated politics over U.S. health care reform (which, ironically, resembles the bill Governor Mitt Romney signed into law in 2005) have transformed Massachusetts into a national battleground. But the dynamics that allow Republican Scott Brown, as extreme as he is unknown, to compete in Massachusetts are not new.