First, you need to understand the difference between spells/abilities and talents. You should generally train all the spells and abilities your trainer offers. I realize money can be tight at first but you'll find it easier to make once you get your professions rolling, and in the meanwhile make training your priority versus buying anything else. Don't bother buying equipment or weapons at the lower levels, just use what drops. Train everything.
Once you reach level 10, you'll get talent points, one per level, that you can spend on different holy, discipline, or shadow talents. This is a completely different thing from training with your trainer, and this is where you specialize in a particular area. Most priests will tell you to spend most of your points in Shadow while you're leveling because it makes for much easier solo play.
Finally and most importantly, at the very top of this forum is the Priest FAQ. Read it. Make it your friend. It'll tell you everything you need to know about the different talents as well as professions and other useful information for leveling up.
ETA: Morghast's reply and your second post came up while I was typing this one. You can level Holy (I am), but it's harder and works best if you're grouped up as often as possible, so hopefully you work and play well with others. :) Again, read the FAQ for Holy talent builds. There was also a thread a while back on the order to spend your points while leveling up. I'll see if I can find it for you.
Edited again to add: Found it. Check out Ixidoria's post here:
http://wow.allakhazam.com/db/class.html?wclass=3;mid=118669238141358242;num=4;page=1 Edited, Oct 3rd 2007 2:28pm by teacake