If you want cheaper mounts, join a guild and participate in writs till its level 30. As your guild progresses, new mounts open up and mounts become cheaper (
Click here for source).
Otherwise, save up every copper and make those platinum pieces; the cheapest horse is still a number of plat.