In the formative stages of Barack Obama’s fame, no topic would make him or his aides more uncomfortable than his personal safety. When I first broached the subject, in July 2004, it was uncharted territory for the then-candidate for the U.S. Senate. Obama had just returned from the Democratic National Convention in Boston, where he had catapulted overnight into the zeitgest of Democrats across the country with his stirring keynote address.

Back on the campaign trail in Illinois, Obama was met by adoring crowds that often ran into the thousands, and his reception was more rock star than political candidate. But with that fervor came some uncomfortable issues—he was eliciting deep emotions, both positive and negative. Suddenly, he was a celebrity, and his rise as a political messiah had begun.

Traveling in central Illinois in the campaign SUV, I asked him if he had thought about implementing extra security precautions. Before he could answer, his wife Michelle, who was seated beside me, cut him off. She said this was a matter that her husband should not be worrying about, that it was for his advisers and staff to handle—and she shot a glance at his communications director, Robert Gibbs.

I didn’t know that Michelle had been discussing this very issue with the campaign team, to little satisfaction. Campaign advisers did not want to hire security personnel because they didn’t want it to appear that they worried about safety—that the candidate feared his constituents.

Largely because Obama is the first African-American to be a serious candidate for president, his safety remains a major issue. He received early and additional federal protection as a presidential candidate, and it remains intense. So this week, when two young, lost souls who claimed to be skinheads hatched an impossible plan to kill blacks and end the rampage with Obama’s assassination, the matter crept into the public conversation.

I recalled Obama’s trip to Africa in August 2006. An editor for a Nairobi newspaper asked me if Democrats would allow Obama to run for president one day. "Don’t you worry that as a black man in America the skinheads will kill him?" the editor asked. The question came from a newspaperman in a country where corruption is rampant, democracy is fragile, and politicians can be prime targets of violence. But the question was pertinent, even here in the United States, where democracy is far more stable, but also where weapons and anger can be just as plentiful.

I mentioned the concern again to Michelle in December 2006 during my last formal interview with her for my book. I think it’s the last time she gave a candid answer on the subject. "I don’t worry about it every day, but it’s there," she said, adding, "It only takes one person and it only takes one incident. I mean, I know history, too."

About a month later, as Obama geared up for a presidential run, the Obamas appeared on the CBS news program 60 Minutes and Michelle gave a much different response, one that obviously had been vetted by political advisers. She said that, as a black man, her husband could be killed pumping gas into his car at any given time—so she didn’t give it deep thought.

Obama has top-notch federal security surrounding him these days. But as we learned in September 2001, no kingdom is impenetrable. I shudder to think of the damage to race relations that would occur, and the social upheaval that might ensue, if something tragic befell Obama. Even if you disagree with his politics, it is inarguable that he has become a symbol of unity and progress to millions of Americans. It’s left to all of us to hope that the darkest side of our nature doesn’t emerge once again to cut short those idealistic dreams.