Plans to convert an empty Victorian villa which became a girls’ school and then council offices in Carmarthenshire have been turned down again.

Lime Grove House, Carmarthen, dates from the mid-1850s.

Over the years it was altered and extended, used as a school, then a private residence again, and then rented to Carmarthenshire Council as offices until 2014, since when it has been empty.

Its owners, HB Thomas & Son (Carmarthen) Ltd, have tried unsuccessfully a number of times to bring the grade two-listed building back into use as a place to live.

Their most recent application to the council was to convert the main building into nine flats, build three four-bedroom houses and two three-bedroom bungalows in the grounds, and knock down detached laboratory offices, lean-to toilets and a boiler room.

There would be 14 parking spaces for the flats and two spaces at least for each of the houses and bungalows.

A design and access statement submitted on the owners’ behalf said the proposal retained two protected ginkgo trees and therefore addressed the only previous reason for refusal.

The main house, it said, was in a state of disrepair and needed “substantial investment”.

It added that the overall setting of the listed building would not be compromised, with Lime Grove House still appearing at the forefront of the site.

Although the statement said a legal agreement about contributions towards things like affordable housing – known as Section 106 obligations – would be addressed separately, the council has turned the application down saying the applicants had failed to sign such an agreement or pay for an assessment of the scheme’s viability to justify not providing an affordable housing contribution.

Some people living close to the villa objected on the grounds of loss of privacy and increased traffic.

Lime Grove House’s varied history is documented in a heritage impact assessment submitted on behalf of the applicants. It described how the surviving sister of John Davies, who built the main house in around 1855, was said to be one of the wealthiest women in Carmarthenshire. She died in 1870.

The heritage report said: “Her funeral procession was a noted affair, including a hearse and mourning coach drawn by four black horses, the whole procession – including specially attired tenants – taking five hours to reach the church at Llangadog at a pace that allowed the foot mourners to accompany the coaches.”