UNESCO Director-General Irina Bokova said she raised the maintenance issue at the Golden Temple of Dambulla with local authorities during her visit to the island.
The Sri Lankan government is bound by international treaty obligations to protect the cave monastery which is home to 2,000-year-old murals and 157 Buddha statues, she told reporters in Colombo at the end of her four-day visit.
A team of experts who visited the site in central Sri Lanka last year found the site poorly maintained and warned that new constructions have affected its heritage value, she said.
A Buddhist monk at the temple has been resisting Sri Lankan government efforts to take over the maintenance of the temple, but Bokova said it was the responsibility of the Sri Lankan state to ensure that the property was well preserved.
"When the World Heritage Committee discusses a report of protection and preservation, it is the responsibility of the government," she said.
"We are talking about an international, legally binding document. The state party to the convention is the government of Sri Lanka."
The cave temple is one of eight World Heritage sites in Sri Lanka and Bokova said she visited three of them.
UN officials have expressed fears that the cave temple could be struck off the prestigious list of heritage sites unless immediate corrective action is taken to preserve frescoes and remove the new construction that has been added by resident monks.
The first site to be removed from UNESCO's World Heritage List was the Arabian Oryx Sanctuary in Oman in 2007 after the government decided to reduce the size of the protected area by 90 percent -- in contravention of the heritage convention.
The only other site to be removed was Germany's Dresden Elbe valley due to the building of a four-lane bridge in the heart of the cultural landscape which meant that the property failed to keep its "outstanding universal value as inscribed."