The Lothingland peninsula is by no means East Anglia's most attractive area. Sometimes, it feels as if the shadow of Great Yarmouth's dismal suburbs is casting a heaviness across the meadows and lanes, and it is always a pleasure to be travelling south away from it. In 1974, they moved the border, putting all that cold suburbia into Norfolk, which is welcome to it. Now, Suffolk starts below the Haddiscoe to Yarmouth road, and one of the very first churches you come to is the pretty round-towered St Margaret at Herringfleet. To be honest, I wasn't expecting it. It was years since I had last been up here, and as we headed down from Fritton I thought that Somerleyton church would be the first to come into view. For a moment, I was disoriented, and then I remembered. Eight years previously, on a fresh spring morning, I had cycled from the opposite direction. This is the only area of East Anglia where most of the parish churches are kept locked, so I wasn't expecting it to be open, and it wasn't. But there wasn't a keyholder notice either, which was a bit worrying. This church has a fabulous collection of stained glass, and, as Ecclesiastical Insurance reminds us, a church which is kept locked all the time is much more likely to be vandalised than one which is open during the day.

Perhaps the parish has taken note of this, because when we came here in May 2008 we found this lovely little church open, and we were able to step into its cool, calm interior. There was a substantial restoration here quite early in the 19th century at the behest of John Francis Leathes, the Lord of the Manor. The restoration was more substantial than at might first appear, since the old window tracery is said to come from the ruins of St Olave's Priory, out on the Waveney edge of the parish. As part of this restoration, the collection of glass was put in place in the east window and on the south side of the chancel. There is a huge quantity of it, with some fascinating details. The great majority appears to be 16th and 17th century continental glass 'rescued' from France and Flanders at the time of the French Revolution - in reality, of course, and as at the time of the English Reformation, much of it was sold abroad by the 'reformers' to raise cash. There is some East Anglian glass among the continental work, which may well also have come originally from St Olave's Priory.

Some of the glass is interesting because it is so unusual, but more unusual than any of the medieval and continental work is the scattering of panes by the Lowestoft artist Robert Allen. Allen owned a glass factory, and produced 'picture glass' at a time when almost nobody else in England was doing it, the end of the 18th and the start of the 19th century. Because of this, he work is entirely pre-ecclesiological - that is to say, it comes from a time before the Oxford Movement had led the 19th century Anglican revival which led to the restoration of churches and the installation of thousands of stained glass windows across the country, mostly of biblical and devotional subjects. The other major figure in this restoration was Samuel Yarrington, the Norwich glazier, who probably arranged and set it all, and may well have been the agent who sold the glass to Leathes in the first place.

The idiosyncratic character of St Margaret comes from the unusually early date of its restoration, but also from the overwhelming influence of the Leathes family. In medieval times, this entire parish was in the ownership and care of St Olave's Priory, and after the rape of the monasteries by the Tudors and their cronies, the ownership of the lands and church passed through various hands until reaching the Mussenden Leathes family in the 18th Century. At the time of the dissolution, the living had been impropriated, which is to say that ministers here were no longer presented by the owners to the Norwich Diocese, but the church became a donative - effectively, a chaplaincy, and the family that owned it employed a minister on a contractual basis for a wage. He was employed by them, and so under the circumstances it is not surprising that the Leathes influence is profoundly felt here.

Wandering around the graveyard, I found a gorgeous terracotta cross, a perfect example of the best that the Arts and Crafts movement could produce. The Morning Stars Sing Together, it says at the top, and So He Bringeth Them To The Haven Where They Would Be. Somewhat surreally, the angel at the top of it appears to be holding a Mercedes Benz insignia. It made me think of that craze in the late 1980s, when what we have now learned to call chavs would wander around in their white shell-suits with a Mercedes Benz insignia hanging on a chain around their necks, in imitation of the Beastie Boys. In fact, the circle divided into three is an ancient Eastern symbol of the Holy Trinity. There was apparently no other inscription, and it took me a moment to realise that the cross had sunk into the soft Lothingland ground. I dug down in front of it, and about six inches below the surface I found a perfectly preserved inscription to a young girl who had died in 1906. Fearing that the the hole I had made might fill up with water and damage it, I covered it up again, but it would be nice if the cross could be raised back to its proper position.