That's assuming that they are voting purely for themselves.
If someone voted to remain because they were concerned about the poorer members of society, you wouldn't care how old they were.
If an old person votes leave, they may be thinking about future generations and how they will be affected in the future.
Edit: Man, that didn't come out as good as I had it in my head...
Basically everyone has the right to vote, whether it be for selfish reasons, or not.