That's pretty much it in a nutshell, hunters (especially pre-BC) are notable for leveling very easily without really ever learning the group dynamic of play, so every hunter joining a PuG was often the wildcard. Like many of the above posters have said, when hunters fail, they fail big. As soloable classes go, I believe that from launch to this day there has been no contender for an easier class to level up, not even close. This class takes off right around level 10 when the pet gets introduced and they don't even really have to spend a lot of time learning it. Outlands of course brought a little bit of a help to this matter by squishing the weak and the lazy but Hunter's are still an easy class for leveling purposes. On the other hand, in PuG's and Raids, they can be one of the most complex classes as players used to soloing integrate themselves into the group style of gameplay. Hunters, with their ease of pulling aggro, lack of (early) group utility, and only so-so cc, have brought in the less experienced players (both in game and RL) with poor social skills to coin the phrase "Huntard". The Hunter ninja types also added to the stigma, with Hunter's being able to utilize a wide variety of weapons and armor, a lot of needlees "needing" occurred and then more players got irate.
It got to the point (pre-BC) that it was so bad that I absolutely gave up playing the class because nobody would accept hunters in PuG's anymore. Post-BC, on a newer server, a lot of people who liked hunters seem to have resolved to correct the issue and L'd-2P. I'm grateful for this because I always loved the class and love group play as well. Now I'm back on hunters again and have taken it upon myself to help keep the rep going north. No other class I know of can calmly sit back and watch an entire "Oh Crap!!!" situation developing while still continuing to burn down enemies and bring things back under control. Tanks flat out have a lot of mobs to keep aggro on in some pulls, and if he doesn't notice a mob break off to pursue an overzealous dps he may not stop it in time to prevent it from having a few squishy snacks. That can be a hunter's time to shine, the pet, the hunter, and his friendly traps can contain a situation very quickly when timed right. Give the tank time to get in position and re-establish aggro. Mages are about the only other class capable of these emergency response procedures of its magnitude with a well-timed frost nova, and those don't last very long. These days I've had no issues getting invited into PuG's and I think it has a lot to do with those who manage to pull off these emergency saves (including myself on occasion). If your server still harbors the same prejudices as before it may just be that people are either too hard-headed or just haven't seen a real hunter in action.
Words of advice for those who have not yet become familiar with the group dynamic:
When the time comes to roll for gear, it is especially essential to be considerate of the other classes before rolling "Need". If it's ranged, you almost get it by default, otherwise, it pays off occasionally to take a hard look at the stat bonuses and consider who else may need it. Ask if you're not sure if the players are ok with you rolling. If, for instance, at level 41, you roll on a set of leather shoulders with good bonuses to agi and AP, your group's rogue or feral druid may get annoyed. Hopefully I'm preaching to the choir, but our class is infamous for these types of situations and you can read up quite a bit on it in the older forums.
And for heaven's sake, take the pet off of Aggressive!!