(CNN) - Former Defense Secretary Donald Rumsfeld's comments earlier this week that a "trained ape" could have done a better job securing diplomatic relations with Afghanistan than the Obama administration sparked a backlash on social media.
But a Rumsfeld aide notes that he has used the phrase before and wasn't referring specifically to President Barack Obama.
"He feels strongly about how poorly the administration has handled diplomacy and that's what was reflected in his comments," Keith Urbahn, Rumsfeld's former chief of staff told CNN Wednesday.
"From a botched 'reset' with Russia, failing to secure a status of forces agreement to Iraq, to alienating Karzai to the point where he won't even talk to U.S. diplomats, to the latest diplomatic fiascos with Syria and Iran, it's been one epic fail after another," Urbahn added.
Rumsfeld told Fox News on Monday that "A trained ape could get a status of forces agreement" with Afghanistan.
"It does not take a genius. And we have so mismanaged that relationship," he said appearing on "On the Record with Greta Van Susteren."
Some on Twitter blasted Rumsfeld's comment as a reference to a derogatory term used against African Americans. Others took issue with Rumsfeld's own foreign policy and leadership of the defense department under former President George W. Bush's administration.
Rumsfeld comments come after Afghan President Hamid Karzai backed Russia's annexation of Crimea, following a referendum in the semi-autonomous region of Ukraine with a large pro-Russian population to secede and join Russia. The United States and members of the G7 condemned both the vote and Moscow's annexation of Crimea.
Rumsfeld said it's "understandable" that Karzai supported the much-criticized referendum given U.S. leaders' public criticism of his leadership.
"It seems to me they pushed him in a political box where he really has very little choice," he said of Karzai.
"I personally sympathize with him to some extent. Nobody, likes to hear a foreign leader side with Putin on the Crimea that way he had. But, I really think it's understandable, given the terrible, terrible diplomacy that the United States has conducted with Afghanistan over the last several years."
Taken as a whole, Rumsfeld's comments appeared to be referring to the Obama administration's foreign policy, not singling out the President, many took his "ape" comment as an offensive reference to the President's race.
Urbahn points to other instances when the former defense secretary used the "trained ape" phrase with regards to foreign policy matters.
In a July 2001 press conference in Washington on missile defense, Rumsfeld used the phrase in talking about how easy it is for countries to use new weapon technologies.
"...with the end of the Cold War and the relaxation of tension in the world, we've seen that proliferation of these technologies is pervasive. And that means that a trained ape can figure out that over the coming period, more people are going to have exceedingly powerful weapons, weapons more powerful than ever in the history of the world, biological weapons, nuclear weapons, chemical weapons," he said, according to a defense department transcript.
Rumsfeld later joked about the ape reference:
"You mean this isn't off the record? Strike when I said 'trained ape.' An untrained ape should know that," he said.
More recently, Rumsfeld used the phrase in an interview earlier this month with conservative radio host Mark Levin to criticized the Obama administration's standing in the international community.
He blasted Hillary Clinton's tenure as secretary of state, again saying a "trained ape" could get a status of forces agreement in Iraq or Afghanistan.
Rumsfeld served as defense secretary under former President Gerald Ford from 1975-1977, and a second time from 2001-2006 under Bush.
In 2002, he also used the phrase in making the case for war with Iraq.
"There's no debate in the world as to whether they have those weapons. We all know that. A trained ape knows that."
In the interview with Fox, he said: "Our relationship with Karzai and with Afghanistan was absolutely first rate in the Bush administration. It has gone downhill like a toboggan ever since the Obama administration came."
CNN's Dana Davidsen, Paul Steinhauser and Nancy Baker contributed to this report.