(CNN) - Republican presidential candidate Mitt Romney is dropping a new addition to his stump speech after the family of the deceased man he mentions in the story raised an objection to it.
A campaign official told CNN that Romney will no longer use the anecdote, which he introduced on Tuesday while campaigning in Iowa and recounted again at a town hall meeting in Ohio.
– Follow the Ticker on Twitter: @PoliticalTicker
– Check out the CNN Electoral Map and Calculator and game out your own strategy for November.
Romney said he and his wife accidentally stopped in at a party in his neighborhood, but the mix-up led them to meet "some wonderful people," among them former Navy SEAL Glen Doherty.
"He came from Massachusetts, where I'd been governor, had family there," Romney said at his Mount Vernon, Ohio event on Wednesday. "He also had skied in some of the places, snow skiing that I had found during the Winter Olympics in Utah, that I'd skied at. And we had a nice chat together."
"You can imagine how shocked I was to learn that he was one of the two Navy - former Navy SEALs killed in Benghazi just a couple of weeks ago," Romney continued. "And I read on CNN International that when the report came that our consulate had been attacked, that he and the other SEAL that was killed with him, that they were in a different place, they were about a mile away, an annex somewhere else in the city. And when they heard that the consulate was under attack, they went to the attack. They didn't hunker down and hide themselves."
Romney did not mention President Barack Obama in his story, but Doherty's mother said in an interview with Massachusetts television station WHDH that she was uncomfortable with the GOP candidate telling the story.
"I don't trust Romney. He shouldn't make my son's death part of his political agenda. It's wrong to use these brave young men, who wanted freedom for all, to degrade Obama," Barbara Doherty, told CNN affiliate WHDH.
His sister separately told CNN affiliate WCVB that she did not have a problem with Romney's story.
"It was nice what he said. Being an American hero is bipartisan," said Doherty's sister, Kate Quigley. "I am happy for it, and I was honored by Obama's coming home ceremony."
The Republican presidential candidate told the story at two events on Tuesday – one in Iowa, and a second in Ohio – and once Tuesday morning. All three events were before the WHDH report received national attention.
Romney campaign spokeswoman Andrea Saul gave CNN a statement on Wednesday morning.
"Governor Romney was inspired by the memory of meeting Glen Doherty and shared his story and that memory, but we respect the wishes of Mrs. Doherty," Saul said.
The mother of Sean Smith, another American killed in the attack which claimed Doherty’s life, will speak out for the first time on CNN’s “AC360” on Wednesday at 8 p.m. and 10 p.m. ET.
– CNN Political Producer Rachel Streitfeld contributed to this report