I will make this simple. Most of the best druid caster leather is either a leatherworking recipe or badge of justice reward. This is if you are limiting yourself to staying in leather gear. I am rather proud to say that as a balance druid before I stopped playing I was in almost all leather for raiding. In this gear I was managing +934 to spell damage and a 18.5% crit chance before moonkin form and any buffs. So it is possible to reach high levels of spell damage in leather but your main source of this gear is not from raids. The raids will get a moonkin his tier sets which are very good in thier own right. If you fell like seeing the gear I was using take a look at Mandragal on the Kirin Tor server. His gear is still showing right now and I may soon be rejoining WoW. Sometimes one just needs a break.
The cloth items that one picks up are to make up until leather can be found. I see no issue with a moonkin druid going after cloth items either. They are very good for spell damage and spell hit. That is a point which most spell damage leather does not get is the spell hit. All casters need this to be successful in raiding.
I will say that your raid leader seems a bit insane with his opinions of what a druid should do. He has obviously never played such a flexible class and does not wish to realize the true potential of what they can do if properly geared and used in the boss killing strategies. But that is neither here nor there. I do somewhat agree that the clothies should have access to those items first only in the sense that that is the only gear they can use for armor. Every guild I have been in has had the "rule" that depending on armor type, the class that uses that type as the best it can use gets first dibs. If none of those classes wants it then the other classes get fair game access to it.
I will also include one final bit, There are a limited number of players on these raids. Given enough time you will eventually get the gear you are seeking. Give it time, be patient and things will eventually work out.