I've been on Zoloft before for my anxiety. It helped somewhat, though I've been trying out new drugs lately. It will take 2 weeks to a month to "kick" in, but I never noticed any side effects (and I do have experience with "peronality-altering" drugs). Your experience may vary though.
There was however an odd sensation when I went off Zoloft, a kind of buzzing in my brain. Wasn't uncomfortable or anything, just weird. I've heard of this in a couple other Zoloft-takers. Just FYI if you experience that, I suspect it's pretty harmless, and it went away after a week or two.
I don't know your case well enough to suggest anything, but it sounds like you're doing the right thing--thinking of various options to try out. I hope the non-drug methods work, but if not, there's really no shame in trying medicine. If you're worried about any permanent changes to you from taking Zoloft, research it well before you do. There are some message boards for this type of thing if you search. My personal experience is that Zoloft doesn't cause any permanent change. Good luck with everything!