The Sun-Times’s Andy Ihnatko is one of the most thoughtful tech writers out there: he’s knowledgable and not afraid of introducing technological details to readers, but also good at translating it to the general reader. So I was glad to see him take on the Romney campaign’s big Orca fail (Orca was the Romney campaign’s voter-database operation, so named because orcas are the natural predators of narwhals, which the Obama campaign’s operation was named after).
Prior to reading his piece, I’d been a bit skeptical about how bad a failure it was; the Obama team knew for four years it would be running a presidential campaign; the Romney campaign knew this for a few months. Reportedly, Narhwal was running when Romney was just clinching the nomination. Having watched vastly less sophisticated projects devolve into chaos over similar timeframes, I had some sympathy. But yeesh:
The login IDs and passwords provided to statewide volunteers were incorrect, and barred them from accessing the app. The campaign didn’t even release the app until election day. They didn’t even release its 60 pages of documentation until the night before…so nearly 40,000 people needed to get up to speed on ORCA at the moment they needed to actually start logging data with it.
And! They were given a URL for ORCA that pointed to a nonexistent http//: address, instead of the correct https:// one.
This is why I like Ihnatko: technical detail.
“They originally had a load balancer and a bunch of app servers, but for some reason couldn’t get it to work properly,” Chris hypothesized. “So they tore out the load balancer, which probably handled the http->https redirect, and forgot to add it back to the now-internet-facing app server.”
A bit more detail from Ars Technica:
Part of the issue was Orca’s architecture. While 11 backend database servers had been provisioned for the system—probably running on virtual machines—the “mobile” piece of Orca was a Web application supported by a single Web server and a single application server. Rather than a set of servers in the cloud, “I believe all the servers were in Boston at the Garden or a data center nearby,” wrote Hans Dittuobo, a Romney volunteer at Boston Garden, to Ars by e-mail.
Orca was supposed to give motivated volunteers access to critical information to assist day-of GOTV efforts. The Web server was like a bridge to get across to the database. If Ihnakto’s source is correct in his theory, the bridge was built incorrectly, and when they fixed the structure they cut off access across the bridge. That’ll happen when you have to rush a project with so many different pieces. But Ihnatko argues that the Romney campaign had simple, out-of-the-box options available to it.
Romney ran on the competence that only a lifetime of private-sector, data-driven, consultant-turnaround experience can bring. The flipside to that is plenty of people in the private sector have been saddled with the computer systems that high-priced consultants leave in their wake. They don’t always work.
Sasha Issenberg, the best-informed journalist on the science and data of contemporary campaigning, has a compelling argument for why Democrats are outperforming Republicans right now in the sophistication and complexity of their GOTV efforts. Issenberg wrote a book about Rick Perry’s gubernatorial campaign in Texas, and the unlikely academics who assisted the governor. How’d a rootin’-tootin’ anti-intellectual get the geeks on his team?
The reason Perry developed that partnership is that he made them an unusual offer, which is that they could publish their work. Most campaigns want to keep it proprietary, so the academics who are willing to work with them are often people who are aligned with their political goals, and not necessarily in it for research purposes.
Issenberg talked to polisci prof Daron Shaw, a Republican who aided Perry’s campaign for reasons other than research kicks:
With an eager pool of academic collaborators in political science, behavioral psychology, and economics linking up with curious political operatives and hacks, the left has birthed an unexpected subculture. It now contains a full-fledged electioneering intelligentsia, focused on integrating large-scale survey research with randomized experimental methods to isolate particular populations that can be moved by political contact.
“There is not much of a commitment to that type of research on the right,” says Daron Shaw, a University of Texas at Austin political scientist who worked on both of George W. Bush’s presidential campaigns. “There is no real understanding of the experimental stuff.”
The advantages of incumbency can’t be dismissed, but the GOP’s data denialists are crippling them. The ill-considered, ignorant backlash against Nate Silver (and the many, many other poll-watchers who had Obama’s probability of winning in a similar, 70+ percent range) revealed a party-wide willingness to double down on bad hands. The most stunning moment came after Obama clinched Ohio: a report that Mitt Romney hadn’t written a concession speech.
People are superstitious; even eminently logical people can be superstitious, and it’s fine—ritual is as much habit as it is belief. But Romney came into election night with the odds against him. They didn’t dictate that he would lose, but they did suggest he need a Plan B, and that it was ultimately more likely than Plan A.
Photograph: BU Interactive News (CC by 2.0)
13 hours ago