James Harris (1709–1780), Fellow Commoner