The four men - identified as Nawab, Ruzi Khan, Mohammad Rahim and Mir Hamza - were being questioned, the sources said.
Afghan police said yesterday that they had arrested two militants of the Haqqani network for the murder of Banerjee, whose book about her dramatic escape from the Taliban was made into a Bollywood film.
They said the two men, arrested in Sharan, the capital of Paktika province late on Monday, had confessed to killing her.
There was a protest in front of Kabul University from 1.30 pm to 2.30 pm today to condemn the killing of Banerjee in Paktika province last Wednesday.
Over 100 people, a majority of them women, joined the demonstration and denounced the killing as inhuman. They also demanded respect for women.
Banerjee, 49, was dragged out of her Afghan husband's home in eastern Paktika province by masked gunmen and shot dead.
Two of the arrested men confessed that they were ordered to kill Banerjee because the film that was made from her book was an insult to the Taliban, Afghan police officials said.
The Taliban last week denied involvement in Banerjee's murder.
Her book "A Kabuliwala's Bengali Wife", an account of her escape from the Taliban two decades ago, was made into a Bollywood film in 2003.