I've sunk more time into Destiny than most but it hasn't really bothered me. I never did anything like shooting the loot cave, and quickly stopped doing things I found dull like patrols. I mostly play it for the social aspect, ie, people to talk to. I find that whenever I'm playing with the clan, it doesn't really matter what I'm doing, it'll be fun.
That said Destiny hasn't gotten in the way of any other games. I got a tiny bit bored with it at one point so I stopped right away and played Far Cry. Then I decided to make a new alt on Destiny and the fun came right back.
I have bought a couple of DS games I've neglected but I decided to save them til the New 3DS comes out, so it feels like I've got more for my new system.
It really is a case of making sure the game is fun, instead of going til you've done everything. I've made the mistake in the past of getting fed up with a game because I insisted on completing it on the hardest setting available first, to save achievement collecting time later. Not worth it. Play it for as long as you're having fun and move on!