Gen Kayani, the longest serving army chief under a civilian government, passed the baton of command to 57-year- old Gen Sharif at an impressive ceremony held at the General Headquarters (GHQ) in Rawalpindi.
Gen Sharif was chosen by Prime Minister Nawaz Sharif as the 15th chief of the Pakistan Army on Wednesday.
The army chief is considered to be the most powerful person in Pakistan, with the military having ruled the country for more than half its 66-year history.
Gen Sharif, who holds the Hilal-i-Imtiaz award, is the younger brother of highly decorated Major Shabbir Sharif, who was killed in the 1971 war with India.
He had superseded senior most military officer Lt Gen Haroon Aslam to the post. Aslam took early retirement and tendered his resignation yesterday.
Gen Sharif assumes charge amid tensions on the border with India and Pakistan's fight with Taliban insurgents and increased violence in the country.
Pakistani analysts view Sharif, who now heads the 600,000-strong army, as a moderate who sees the militant threat inside Pakistan as just as important as the strategic tussle with India.
Before his last posting as a Principal Staff Officer in General Headquarters, Sharif commanded the Gujranwala-based XXX Corps, which is responsible for the Line of Control and the international border in Punjab, between 2010 and 2012.
He is considered an old India hand and played a key role in framing Pakistan's response to the Indian Army's new doctrines, sources said.
Though Prime Minister Sharif has pressed for peace with India, it will all depend on what Gen Sharif really wants.