Romney says wealthy donors ‘by and large…doing just fine’

Southampton, New York (CNN) – At an exclusive gathering in Southampton, Mitt Romney told wealthy donors he worries more about helping the poor and middle class rather than wealthier individuals there, including himself, who he believes are “by and large…doing just fine.”

The presumptive Republican presidential nominee broke away from a weeklong vacation with his wife Ann and their large extended family at the Romneys' New Hampshire home on Lake Winnipesaukee to attend a trio of fund-raisers in the Hamptons, the ultimate summer resort of the rich and famous.

- Follow the Ticker on Twitter: @PoliticalTicker

- Check out the CNN Electoral Map and Calculator and game out your own strategy for November.

As Democrats continued to hammer Mitt Romney for his vast personal wealth and handling of finances that includes offshore bank accounts, the White House hopeful raised millions on prime real estate of billionaires Ron Perelman and David Koch, and former U.S. Ambassador Clifford Sobel.

The reception and dinner in Southampton and lunch at Perelman’s estate in East Hampton were private events and closed to reporters. During the reception in the afternoon, Romney was speaking outside when CNN and another reporter overheard his remarks to people gathered under a tent on a tennis court.

“By the way, you guys are doing fine,” Romney was heard saying at Sobel’s beachfront home. “If you’re here, by and large you’re doing just fine. And I don’t spend a lot of time worrying about those that are doing as well as you guys are, or how I’m doing, but I spend a lot of time worrying about those that are poor and those in the middle class that are finding it hard to make a bright future for themselves.”

Early in the GOP primary campaign, Romney had to clarify a comment he made on CNN about not being “concerned about the very poor.”

He said his words were taken out of context and explained his campaign’s focus was on the middle class, stating that there are programs to help the poor and the rich are doing well on their own.

At the Southampton reception, Romney said “Democratic friends are always taking the high road,” casting his fellow Republicans as people who don’t care for anyone but themselves.

“We all care about the poor, we want to help the poor,” Romney said to applause. “But what we want to do is not just help; we want to help people from becoming poor.”

"Wealthy people are doing fine," Romney told reporters in February. "But my focus in the campaign is on middle-income people. Of course I'm concerned about all Americans - poor, wealthy, middle class, but the focus of my effort will be on middle-income families who I think have been most hurt by the Obama economy."

Romney, who told at least 100 people in the Hamptons he believed they are doing fine, has taken President Barack Obama to task for stating in a recent interview that the “private sector is doing fine.”

Many donors who arrived in top-of-the-line luxury cars to see Romney and his wife were greeted by some 200 protesters, gathered near the dinner hosted by Koch, with a large sign that read, “Romney has a Koch problem.” A plane flying overhead carried a banner with the same message.

– CNN’s Rachel Streitfeld and Jessica Yellin contributed to this report.