People trying to talk to you during a set.... (kinghelmet)

This happens so much at my gym and personally i think its so irritating. Smashing through a set, headphones in, then some chap comes over and decides to try and have a discussion with me? Is it just me or does this **** other people off too? Sorry for the..