The reason I haven't mentioned them is because I must admit I don't have the slightest clue about their racials, aside from gift of the Naaru which is fairly useless for a priest PvP-wise. I've looked it all up for you now and here's a large post to summarize it all;
Human
-Desperate prayer, an instant (at 70) 2K heal free of mana cost. Life-saver in PvE and PvP
-Feedback, a 15 second buff which mana burns attackers' energy. Useless in PvE, quite helpful in PvP.
-Perception, a buff that pretty much negates stealth. Useless in PvE, very powerful in PvP against rogues and druids (obviously).
Dwarf
-Desperate prayer
-Chastise, a very fast casting spell with a small stun. Useless in PvE, quite useful in PvP as a cheap and fast interrupting spell.
-Stone form, immunity to poison, bleed and disease effects. The disease effects immunity isn't too great since you can dispel diseases anyway. On the other hand, poison immunity is a life saver against rogues and bleed against warriors. Extremely useful in PvP, fairly useful in PvE.
Night elf
-Starshards, a free 15 second weak DOT. Useless in PvP and PvE. In PvP, if you need a quick DOT on an enemy (to for example prevent a rogue from restealthing) you should use rank 1 Shadow Word: Pain. In PvE, you really don't need this.
-Elune's Grace, a weak form of the rogue's evasion. Fairly useless in PvE (if you're getting hit this isn't going to save your life, though it's helpful), avarage in PvP; it helps against melee classes and hunters, but it's not exactly life-saving either.
-Shadowmeld, stealth as long as you stand still. Next to useless in PvE but extremely useful in PvP, both BG's and arenas. Even if you can't move, it will lead enemies to think there's a rogue/druid (in the arenas) or there is nobody guarding that objective (in the BG's), allowing you to jump in whenever needed. The power of the element of surprise is NOT to be underestimated.
Draenei
-Symbol of hope, a 5 minute cooldown spell that basically replenishes some of your party's mana. Fairly useless in PvP as you rarely need that mana boost, though it has it's use in the arena. Quite useful in PvE, mainly because it replenishes yóur mana as healer... In raids it's even quite lovely.
-Chastise
-Gift of the Naaru, a small heal-over-time spell. This is fairly useless for a priest. It heals a bit less than your renew at level 70, and while it's free of mana cost, there's a 1,5 sec cast timer. Alright, maybe it isn't useless, but you aren't going to use this much in PvE because you have plenty of other healing spells available. For PvP it is mostly too weak to have much impact, but can't be neglected; the short cast time makes it PvP viable and combined with renew you've just placed a fairly strong HOT on your target.
The end result of this is that for PvP, my top 4 alliance priest races would be
1) Dwarf
2) Human
3) Night Elf
4) Draenei
The thing is that the Dwarf and Human racials (mainly desperate prayer) increase your survivability by quite a lot, which is what you are looking for in PvP. Dwarves do this better than Humans mainly thanks to Stoneform. While the Night Elf shadowmeld is incredibly powerful, their 2 other racials are useless for PvP. Draenei simply come last. While Chastise is nice, Gift of the Naaru is only avarage for PvP; if you want a Draenei you might as well pick a Dwarf and benefit from better racials.
For PvE (emphasis on raiding), my top 4 alliance priest races are
1) Draenei
2) Dwarf
3) Human
4) Night Elf
The Human and Dwarf racials that increase survivability now work against them. In PvE as a priest, you do not need survivability. Draenei reign supreme in this field of play. Their +hit aura doesn't benefit you too much, but will assist all casters in your group in a raid. We also see Symbol of Hope going from useless to "exceptionally powerful", with gift of the Naaru as a free extra HOT during spike damage. Night Elves rank poor because their 'survival racial', Elune's Grace, simply does not beat the instant free heal Desperate Prayer gives. Dwarves still are still standing solid on the second spot. While not as good as Draeneis, their Desperate Prayer has a better chance of allowing you to live if the tank loses aggro than Elune's Grace. Furthermore, stoneform finds use in a number of raid encounters, whereas the Humans' Feedback is utterly useless.
So to sum it all up; if you care about the stats I say Dwarf is the way to go. They reign in PvP and stand their ground for PvE. Since you said you were after PvP mostly, Draenei is what I'd least recommend. You could however like you said roll a Night Elf and do the Draenei starting quests; while the other two Night Elf racials are weak, Shadowmeld is probably the most powerful PvP racial. Last but not least, remember this is just a game; if you like Draeneis, then ***** me and roll a Draenei. In the end the racials matter fairly little.