When Bishop Pompallier, a Frenchman, visited his compatriots in Akaroa before the end of 1840, he consecrated this ground as a cemetery. It is thought that over the next few decades some 80 people were buried in the cemetery.
In the 1850s and 1860s, public cemeteries were opened at the other end of town (see Town Cemeteries), and the number of burials declined, but people with family members already interred in the French Cemetery were buried there possibly as late as 1880. The disused cemetery then fell into disrepair and became overgrown. The wooden crosses that marked the early graves disappeared, but metal plaques mounted on two of them survive.
In 1925, the overgrown cemetery was cleared and the surrounding concrete wall and central memorial were erected. The names of people known to have been buried in the cemetery were inscribed on the memorial, though the list is probably not complete or entirely accurate. Now surrounded by fine old trees and attractively planted, the cemetery is a calm, restful place.