Nineteen Royal Australian Navy (RAN) and two British Royal Navy personnel died when the converted ferry HMAS Kuttabul was sunk by a Japanese midget submarine just after midnight on June 1, 1942.

The submarine, known as M24, was one of three midget submarines that entered Sydney Harbour that night.

It fired torpedoes at the cruiser USS Chicago but missed and instead hit HMAS Kuttabul.

The submarine then disappeared until 2006, when scuba divers discovered its wreck off a Sydney beach.

The attack will be remembered as a time when war came to Sydney Harbour, Commander Todd Wilson from the RAN told the 71st memorial service at Garden Island in Sydney's east on Friday.

"It was a night in history where the city and nation's perspective that our geographic isolation would protect us was shattered," he said.

RAN Commander Lachlan King's father, Scott King, was one of the remaining survivors from the attack.

"They were just young men, and I think it's important that we remember the sacrifices they made.

"I don't think my father was the free soul that he may otherwise have been had he not had to experience that sort of tragedy," Commander King told AAP.

The ceremony also recognised all six Japanese submariners who died.

"That's an important part of the memorial," Commander King said.

"The reconciliation of our two countries and the fact that with reconciliation, we avoid the necessity for great tragedies as that were experienced here in Sydney Harbour."