"I'm the president and I'm always responsible, and that's why nobody's more interested in finding out exactly what happened than I do," Obama said during the 90-minute face-off at the Hofstra University here for a town hall style debate.
His comments came a day after Secretary of State Hillary Clinton took "responsibility" for the security failure in defending an attack on the Benghazi Consulate last month that left US Ambassador Chris Stevens and three others dead.
Obama praised Clinton for the work she has done but said ultimately the responsibility for the safety of Americans posted abroad lies with him. "Secretary Clinton has done an extraordinary job. But she works for me," he said.
The "suggestion that anybody in my team whether the Secretary of State, our UN Ambassador, anybody on my team would play politics or mislead when we've lost four of our own, governor, is offensive. That's not what we do. That's not what I do as president, that's not what I do as Commander-in-Chief," an aggressive Obama then told Romney looking directly in his eyes with an angry stare before heading back to his seat.
Obama also accused Romney of trying to use the Libya attack to score political points saying that is not how a President and Commander-in-Chief operates.
"While we were still dealing with our diplomats being threatened, Governor Romney put out a press release, trying to make political points, and that's not how a commander in chief operates. You don't turn national security into a political issue," a visibly testy Obama said.
Libya attack evoked strong reactions and a fiery back and forth between Obama and Romney with the Republican candidate accusing the Obama administration of being unclear for days on whether the Benghazi assault was a spontaneous demonstration over an anti-Islam video or a terrorist attack.