Rick Perlstein is an invaluable historian and essayist of America's right-wing movement, the author of Before the Storm and Nixonland, which I both highly recommend. Like a lot of experts, he came to his expertise through an odd, early obsession:

I'm in a unique position to judge. A sixties obsessive since childhood, I misspent my teenage years prowling a ramshackle five-story used-book warehouse that somehow managed, until last October, to stay one step ahead of Milwaukee, Wisconsin's building inspectors. There, I collected volumes from a decade gone mad: texts by Black Panthers decrying "AmeriKKKa"; by New Leftists proclaiming that "the future of our struggle is the future of crime in the streets"; and by right-wingers like preacher David Noebel, who exposed the "Communist subversion of music" by which Russian spymasters deployed Pavlov's techniques to rot the minds of America's youth via their bought-and-paid-for agents, the Beatles.

Perlstein spent years virtually traveling the byways of American political insanity, which eventually led him to Box 722 at the Chicago History Society, a collection of letters sent to Illinois Senator Paul Douglas during the Civil Rights era:

As a Gage park resident & that of my in-laws & my parents, & their familes we are living as decent, hard-working people, you should consider martial law to prevent a peaceful community from getting harassed. That you should consider re-establishing law &order & change laws to protect the people and not criminals & people who openly voice their opinions against the majority as well as the government. Our children don't get sprinklers, day courts, new schools, elevators, cheap rent, yet they will be asked shortly to go fight on foreign shores. I think its time to defend our country from within. I have 3 sons & I will gladly have them defend this country here.

"Reading them in that archive, I felt like I was peering into the dark soul of America to a depth I'd never thought possible," Perlstein writes—quite a perspective from someone who'd been peering into the dark soul of America since his teenage years in a massive used-book warehouse.

Whenever the subject of website comment threads and the journalists and readers who loathe them comes up, I think about Perlstein and Box 722.

Recently it was Nick Denton, speaking at SXSW. When Denton, creator of the Gawker empire—which hosts some exceptional comment threads, particularly on Jezebel—speaks about how to approach content, it makes news, at least among the bloggerati. And Denton, whose sites maintain above-average comment quality (not a high bar, I realize) is burned out and bummed out:

@nickdenton: the problem w/comments is scale. Tragedy of the commons. Quality deteriorates to a wasteland. #SXSW #ketchum #dentonint

Also bummed out, a journalist who couldn't be more unlike Denton, my former colleague Steve Bogira (in comments):

I'm fine with grouches. Grouches can be great teachers. Royko was no Mother Teresa, and Mother Teresa herself was cranky before her morning coffee. It's the personal attacks and the venom that I'd like to see less of. Irrelevance bothers me too. Back in the day, the need to address an envelope and pay for a stamp were helpful screening factors.

My first official duty in journalism, back in middle school, was reading letters to the editor. So I can attest that the requirement to send a letter was a helpful screening factor, but in order to maintain the relevant, reasonable tone desired by the editorial page, more work was needed, so it was farmed out to me. Practically every print publication has a wall or corkboard that can attest to the frequency of irrelevance and venom floating out there in the world.

Looking back on those handwritten missives to the Roanoke Times & World News, I can't help but wonder if I was participating in something counterproductive in filtering out the atavism. This isn't to say I think Denton is, or anyone else who wishes to raise the discourse in his or her little corner of the Internet. Denton wants a specific reader experience, which I appreciate in my own particular comment haunts, and attempts at automagically fashioning the conversation are often worthy experiments. Plus, atavism will find its place—it did before the Internet, on the airwaves and in mimeographed handouts and Larouche tables outside student unions.

And maybe that's good, as chilling as it is. Journalism is a loose institution predicated on reason, as New York Times reporter Charles Duhigg noted to Ira Glass in addressing the Mike Daisey fiasco. He goes to China, reports what he sees and hears and reads, and presents that to the audience. That's his job; it's the reader's to process that information and act on it. Obviously this process frequently works to the greater good. Duhigg's work has raised awareness of work conditions at electronics plants in China, and is an essential part of the discourse between consumers and corporations. Other journalists, who have the leeway to present their own opinions, can attempt to influence this process more directly by presenting their own arguments and solutions.

But this is not how everyone reacts and interacts with the world. Or most people. Or anyone:

Behavioral economist Colin Camerer (1998:56) recommends replacement assumptions that allow economic agents

to be impulsive and myopic, to lack self-control, to keep track of earning and spending in separate “mental accounting” categories, to care about the outcomes of others (both enviously and altruistically), to construct preferences from experience and observation, to sometimes misjudge probabilities, to take pleasure and pain from the difference between their economic state and some set of reference points, and so forth.

If that sounds obvious, it wasn't all that obvious to the economic super-agents who followed efficient-market theories over the cliff in the last decade. Behavioral economists could learn a lot from comment threads; certainly it would be difficult to not include impulse and myopia in their models after a quick trip through them.

And now the comment threads are moving above the fold. Recently I've seen an increase in what might be termed meta-trolling, or troll harvesting: posts that collect the most abjectly stupid or offensive comments on a hot topic and lay them out there, for the reader to take pleasure or pain from the difference between their intellectual state and some set of reference morons. BuzzFeed, which is less a website than it is a venture-capital firm that deals in the coin of social-media capital, has made a product line out of it:

At a time when massively popular Internet sensations often seem random—irreplicable one-offs such as “Kony 2012″—Stopera produces reliable hits. In recent months his blockbusters have included “25 Extremely Upsetting Reactions to Chris Brown at the Grammys” (2.5 million page views)….

"25 Extremely Upsetting Reactions to Chris Brown at the Grammys" is just that: women saying they'd let Chris Brown punch or beat them because he's attractive/successful/etc. Also in the BuzzFeed line of irrationality harvesting: "10 Incredibly Racist 'Hunger Games' Fans"; "The 25 Most Shocking Reactions to Snooki's Pregnancy Announcement"; "Who the F*uck is Bon Iver/Bonnie Bear!?"; and so forth.

BuzzFeed tends to play off its troll harvesting for that little frisson you get from discovering that there are plenty of people dumber than you are. But some people do it to raise awareness. Charles Johnson is the proprietor of Little Green Footballs, a popular blog that was, for a time, reviled as a hate site for its anti-Muslim rhetoric. At some point, Johnson must have had an epiphany, because recently he's been getting a great deal of attention from sites that used to flame him for his deep-diving expeditions into the comments sections of FoxNews.com. The ones on Trayvon Martin are as bad as you'd expect, but the ones on Whitney Houston are no better.

Johnson's not alone. At Cracked, David Wong uses the comments at Free Republic as a jumping-off point for "5 Ways Men Are Trained to Hate Women." Hunger Games Tweets is a Tumblr devoted exclusively to fans of the book who are upset that one of the characters in the movie (who is is explicitly described in the book as dark-skinned) is played by a black actress. E.g.: Awkward moment when Rue is some black girl and not the little blonde innocent girl you picture. The excellent blogger Sady Doyle started an awareness hashtag, #MenCallMeThings, for women to record the horrifyingly misogynist things the anonymity of the Internet allows men to say to women in the public sphere.

Almost everyone I know disagrees with me on the social value of nightmare-fuel comment threads. And they have a point that putative news organizations should have a moral obligation to approach the discourse they tend with the same values as the one they create themselves. Even if it's just a negative obligation to not profit from racism and misogyny. (How to prioritize and implement that is another issue altogether.)

Which is to say that they have a right, and probably an obligation, to control their internal discourse. What I'm not convinced of is that such efforts have any effect on the external discourse: it's not potty-training, it's flushing the toilet down a different series of tubes.

And thoughtful people like Rick Perlstein can find immense value in that sewage. The pamphlets of Perlstein's childhood have moved online, which has not just allowed more people to participate in the feces-slinging, it's also given many, many more of us the access that Perlstein had in his high-school days. Which, I hope, has made us more aware not just of the anonymous circus of misanthropy, but of the methods of the professional trolls who ringlead it: people like Lee Atwater, who helped define the terms:

In 1981, during the first year of Mr. Reagan’s presidency, the late Lee Atwater gave an interview to a political science professor at Case Western Reserve University, explaining the evolution of the Southern strategy:

“You start out in 1954 by saying, ‘Nigger, nigger, nigger,’ ” said Atwater. “By 1968, you can’t say ‘nigger’ — that hurts you. Backfires. So you say stuff like forced busing, states’ rights, and all that stuff. You’re getting so abstract now [that] you’re talking about cutting taxes, and all these things you’re talking about are totally economic things, and a byproduct of them is [that] blacks get hurt worse than whites.”

Atwater's dog-whistle politics have been open-sourced, which allows a lot more people to do it. But also allows many more than those to see exactly how it works, and that's why Perlstein is standing athwart breaking news, yelling at people to look back at history:

But are right-wingers scarier now than in the past? They certainly seem stranger and fiercer. I'd argue, however, that they’ve been this crazy for a long time. Over the last sixty years or so, I see far more continuities than discontinuities in what the rightward twenty or thirty percent of Americans believe about the world. The crazy things they believed and wanted were obscured by their lack of power, but they were always there – if you knew where to look. What's changed is that loony conservatives are now the Republican mainstream, the dominant force in the GOP.

Perhaps I'm pushing the correlation/causation barrier past its breaking point, but the "epistemic closure" that moderate and/or election-minded Republicans fear is destroying their party from the inside like a brick dropped inside a dryer seems to have hit launch speed during the Web's young adulthood.

This is not a new idea. The great genius of the civil-rights movement was the realization—derived from the philosophy of nonviolent resistance—that you have to tease the hate into the open for people to truly understand its scope, out of Box 722 and into the streets. A lot of people worked very hard, and took enormous risks to do so. In the Internet age, some of that process has become automated.

Photograph: Wikimedia Commons