Handled like a pro, well done Rummy. People like that really aren't worth the grief. Let them feel hard while they're out with their pathetic following because when they get home to their council estate and alcoholic, jobless parents they will remember how ridiculous their existence is.
I'd like to think I would've just turned around and sucker-punched the guy right in the face, bringing him down in one. Truthfully I probably would've handled it the same way as you which, though it doesn't often feel it, is the best way. Avoid confrontation, be the bigger person and just ignore it.
You could always use the Stephen Fry defence with people like that, just say 'don't hit me, I'll get an erection!'.