"Caste-based discrimination remains widespread and deeply rooted, its victims face structural discrimination, marginalisation and systematic exclusion, and the level of impunity is very high," the group of experts noted.
"This form of discrimination entails gross and wide-ranging human rights abuses, including brutal forms of violence."
People who are considered of low caste in South Asia are known as 'Dalits' or 'untouchables'.
In many countries, they face marginalisation, social and economic exclusion, segregation in housing, limited access to basic services, including water and sanitation and employment, and work in conditions similar to slavery.
The experts said that Dalit women and girls are particularly vulnerable and face multiple forms of discrimination and violence, including sexual abuse.
Children are also at high risk of being sold and sexually exploited.
Two years ago, Nepal adopted the 'Caste-based Discrimination and Untouchability Bill', a landmark law that protects the rights of Dalits.
More recently, the British Government decided in April that the Equality Act would cover caste discrimination to protect Dalits in Diaspora communities.
While these are positive steps to eliminate caste-based discrimination, the experts expressed concern about a serious lack of implementation in countries where legislation already exists, and called for an effective application of laws, policies and programme to protect those affected by this type of discrimination.