You mean by the docks/old bank/old mage shop/etc. ?

I tried getting back there using a boat and didnt succeed...cant remember how hard I tried tho.