The explosion resulted in a major fire breaking out on board Sindhurakshak, a Russian-made submarine, shortly after midnight, defence sources said.
TV footage of the incident showed a huge ball of fire after the explosion lighting up the night sky in Colaba area, where the naval dockyard is located.
There were 18 persons on board the 2,300 tonne submarine, powered by a combination of diesel generators and electric batteries, a defence spokesperson said.
Defence Minister A K Antony confirmed the death on board the Kilo class warship INS Sindhurakshak but gave no details on the casualties.
"I am saddened by those naval personnel who lost their lives in the service of the country. It is a great tragedy for the Navy," Antony told reporters in Parliament House before leaving for Mumbai.
Earlier he briefed Prime Minister Manmohan Singh about the mishap. Navy Chief Admiral D K Joshi has also reached Mumbai.
Naval sources said the blast took place in the front portion of the 16-year-old submarine which housed the torpedoes.
The navy has ordered a board of inquiry to probe the explosion and subsequent fire in the submarine, he said.