Just because it's outside doesn't mean it's giving you perfect ventilation. Naturally this would depend on where the smokers blow their smoke, because the ones I see don't aim for the sky. They aim ahead or down, which means it's going to linger not even counting the ever-slow movement of the smoke from the cigarette alone which doesn't dissipate as quick as you may think. The fact is you have to walk through it in many cases. Set your garden on fire and walk through the flames, see if they affect you. Don't worry it'll be outside so it's well ventilated :p
Joking aside, I don't mind those that smoke. If you want fuck your lungs or cancer or whatever they attribute to smoking these days, that's your choice. Power to the people. All I ask is that you don't bring me into the equation.
What got you going? As mentioned previously everyone I've encountered who smokes said they didn't like it to begin with. What made you go back?