As far as I'm concerned, it's what the devs manage to do with the hardware, than any sort of technicalities. I don't think graphics are as clean cut as in how many polys are being pushed, or which game has bloom and such, I think it goes by which game just looks better. On a technical level, Forza is better than GT4, I never denied this, but through dev talent, and "smoke and mirrors", GT4 just looks better, even if it technically isn't.
If you compare an N64 game like V Rally to a game like Gran Turismo, well just because of the fact it was on the N64, V Rally had texture filtering, yet Gran Turismo didn't. Does that make V Rally a better looking game over all?