I too find the apparently innocent and objective appeal to "common sense" frustrating.People often do not realize that their appeals to common sense are based on the norms they have established with other like minded people -- based on gender, class, education, shared world view, history, etc.
For instance, 80 years ago and among some Canadians it was "common sense" that women should not have the vote. C'mon, use your common sense -- that's just giving 2 votes to married men!
I have not found a satisfactory way to deal with the common sensers of the world.