In PvP you leave combat when no action has been taken for 6 seconds. If you spec into Imp Gouge, this gives you a 5.5 second gouge, which should allow you to restealth just as your opponent comes out if you hammer on your hotkey.
In PvE the only way to leave combat is to kill your target or not fight and kite it far enough where it loses interest.
Edit: Fixed durations, thank you Theo.
Edit#2: Fixed again, L2wowhead. Thanks Kavekk
Edited, Oct 9th 2008 2:21pm by AtrophyGFour
Edited, Oct 10th 2008 8:12am by AtrophyGFour