You have to talk to the captain and use his ship to travel to Tempest Bay. If you just port there you can go get your cloak but the quest will not complete.