Revenge of the geeks

In the weeks since the election, I've been slowly weaning myself off political blogs and TV shows. Over the course of a random twenty four hour period in the two months between the convention and the election, I'd keep up with posts on all the standard political sites, and sometimes watch most of MSNBC's triumvirate of Olbermann, Matthews, and Maddow, plus the Daily Show and Colbert Report on Comedy Central. It was almost like having a second full time job!

After the election, I started cutting off the MSNBC shows as well as Jon Stewart, even though I still can't drop the Colbert Report. This is like the political offseason, where there's a bunch of discussion about things that aren't that important, so I've also stopped reading most political blogs, but I'm still regularly checking in to Nate Silver's 538.com. He's even changed their motto from "electoral projections done right" to "politics done right".

Silver's idea to applying his sabermetric analysis to elections was almost as successful in the 2008 election cycle as Barack Obama's campaign. The PECOTA algorithm he developed for Baseball Prospectus is unbelievably accurate in projecting performance of baseball teams (guessing that the Tampa Bay Rays would make the playoffs last year for example), but there was some question whether baseball stats could extend to election polling.

There was some "meta-analysis" during the 2004 election, but it didn't go as deep or make as much impact as 538.com and other sites like Sam Wang's Princeton Election Consortium did in 2008. These sites analyzed the daily polling data in nearly real time to determine the projected win percentages, electoral vote, and popular vote totals for Obama and McCain, and ended up projecting what happened in the actual election.

Sabermetric analysis is often derided by people inside baseball as egghead wankery by geeks in their Mom's basement who probably couldn't hit the curveball. Even when it's practiced by folks like Oakland A's GM Billy Beane who actually played the game. There's a similar prejudice in the political world that these geeks don't know what's really going on -- they're just silly bloggers in their Mom's basement.

Stats have been used with a lot of success inside the sports world (where Bill James is a consultant for the Red Sox, and his sabermetric strategies are still employed by other organizations), but until this year (or perhaps two years ago) political campaigns were still run based on time honored legends and seat of the pants intuition. "Only Big states matter", "Democrats can't win in Virginia", "The Bradley Effect will keep Obama from winning".

When 538.com launched during the Democratic primaries, it was obvious which candidate Nate Silver was supporting (he's a Chicago guy after all), but his model showed no bias. And his objective analysis was just trying to determine who was "ahead" based on who won the latest caucus or primary, but who stood the best chance to win the big contest in November. They determined that Hillary Clinton had no chance long before anyone else did, and that John McCain was on shaky ground, when many experts (including comments here) thought he was sure to win.

538's greatest achievement after the election was throwing a wrench at the meme that California's Prop 8 passed on the backs of Obama's new African-American voters. One exit poll claimed that 75% of AAs "supported" Prop 8, therefore CA's homophobia was all caused by black folks. QED. I was upset by these theories for many reasons, but they all ran out of gas after Silver's post one week after the election showed how misguided that theory was. Statistics don't really "disprove" anything, but they do show that certain things are unlikely, like 6% of the population being responsible for anything based on an unscientific sampling of 6% of that 6% (around 0.36% of the voters).

There's also been lots of analysis of the recounts and runoffs in Alaska, Minnesota, and Georgia on 538.com, as well as interesting stories about Nate's adventures interviewing right wing hacks like John Ziegler. I wasn't sure what would become of that site after the election, but it seems like it's found a niche in the post-election world as "politics done right". Score one for the geeks!