Well it is the conclusion I reached, and the one Jonnas reached...and the one I reckon quite a lot reached. If only because the actual outcome Nintendo told us about later doesn't really make sense, as they never touched on alternate dimensions in the game and it otherwise makes no sense why you would not have just been sent back to the right time straight away.
Then again this is Zelda, so trying to make any real sense of it is all a bit futile. They've tried to sew together a grand story when in all honesty, I doubt they put that much thought into continuity originally.