It actually came from a phrase used by the East India Trading Company, which, of course, was based in London.
When it booked passengers round-trip to India, the more affluent passengers would request a cabin on the side of the ship least exposed to the Atlantic Ocean gales.
Hence, they were given cabins ‘port outbound, starboard homebound.’ It eventually was abbreviated to posh.