Washington (CNN) - When fed up House Democratic leaders called on Anthony Weiner to resign last weekend, they were hoping to keep what they consider a disruptive political distraction from entering a third week.
But Weiner did not step down, and Tuesday outside the House Democrats first meeting since Weiner admitted to sending lewd Twitter messages, frustration among some of his colleagues was palpable.
"It is taking away from all the issues we are fighting for. It's been what two weeks? It's been a very long two weeks," said Weiner's Democratic colleague from New York, Rep. Carolyn McCarthy.
McCarthy told reporters "we're hearing that he might resign in a couple of days," but did not say where she was hearing that.
While McCarthy stopped short of saying Weiner should leave Congress, others did not.
Rep. Zoe Lofgren, who chaired the House Ethics Committee through the end of last year, said "I think he should resign."
So does Rep. Sander Levin of Michigan, who said it is important to send a "strong message to him that he should resign."
"The more of us who say it, the more telling it will be," said Levin.
If Weiner does not resign, fellow Democrats could take other punitive steps such as voting to strip him of his committee assignments or even vote to expel him from the Democratic caucus.
But multiple lawmakers said House Democrats did not discuss those options in Tuesday's meeting, and said the discussion about Weiner was limited to House Democratic leader Nancy Pelosi repeating to colleagues what she said publicly over the weekend.
Afterwards, she reiterated that to reporters.
"I wanted to be sure that they knew why I came to the conclusion that with the love of his family and confidence of his constituents and the need for help that Congressman Weiner should resign from the Congress. I'm never prouder of my colleagues than in times of challenge like this," said Pelosi.
Behind the scenes, Pelosi is expressing confidence Weiner will heed the call from his colleagues to step down.
A source inside Tuesday morning's House Democratic meeting tells CNN that one lawmaker did ask Pelosi about the possibility of stripping Weiner's committee assignments, and that she replied "he won't have any when he resigns."
Weiner has made clear to colleagues and friends that he is waiting to make a decision until his wife, Huma Adebin, returns from a trip abroad with her boss, Secretary of State Hillary Clinton.
One Democratic source who spoke with Weiner Saturday tells CNN he said he wants to "look her in the eye" and talk to her about whether to resign, because he has hurt her enough.
Abedin could be back as early as Wednesday morning, and multiple House Democrats have told CNN they are hoping he will announce soon after that he will resign.
Steve Israel, one of the Democratic leaders who issued coordinated statements calling on Weiner to step down, said Monday that "he's awaiting the arrival of his wife, having a conversation with his wife. I hope he makes the judgment that he should resign."
Weiner announced Saturday he was entering an undisclosed treatment center, and on Monday, was granted a two-week leave of absence from the House.
House Democratic Whip Steny Hoyer stopped short of saying Weiner should resign, but made clear he thinks that would be the best decision, telling reporters it would be "very difficult for him to proceed given the circumstances."
Hoyer added, "I hope he's seriously considering taking a course that will take him out of this context, and deal with this personal life and not be burdened by being a member."
To be sure, some of Weiner's colleagues say it's not their place to pressure him to resign.
"He was duly elected by his constituents I think that's a decision for him to make," said Rob Andrews, D-New Jersey.
As for Republicans, House Speaker John Boehner, R-Ohio, joined other GOP colleagues Tuesday in saying he thinks Weiner should resign
–CNN's Paul Courson contributed to this story