Two deaths were confirmed by Fahim Jahanzeb, the spokesman for the state-run Rescue 1122 service. He told PTI that another 50 to 60 people were believed to be trapped in the debris of the factory at Multan Road.
Witnesses said the bodies of a man and a woman had been found in the rubble.
Officials said the three-storey factory building collapsed after a boiler exploded. Three houses next to the factory also collapsed, they said.
Rescue workers and local residents initially dug through the rubble with their hands as heavy machinery took some time to reach the site due to the narrow streets in the area.
One police official said 50 people had reported for the morning shift at the factory though local residents claimed up to 100 people were in the building at the time of the blast.
Local residents said the factory, which made pharmaceutical products, had been sealed in the past. They said they had complained to authorities for over a decade for the factory to be shifted out of the residential area.
Lahore police chief Malik Ahmad Raza Tahir confirmed to reporters that the factory had been sealed twice in the past.
"Such factories should not be run in residential areas," he said. Footage on television showed the building had been reduced to a huge pile of bricks and twisted metal.
Witnesses said the roofs of two storeys had collapsed following the blast that went off at 8.15 am.