Kishore Kumar had once refused to lend his voice to a song, even after knowing that Lata Mangeshkar will be giving her voice for the female number.