Samuel Harmer (1721/1722–1808), Speaker of the Common Council of Norwich