The collection was started in the 1830s by Augustus Smith of Tresco Abbey, the founder of the gardens, and was moved to its current home in the 1870s. It was acquired by the National Maritime Museum in Greenwich in 1972 in lieu of death duties. The exhibits mostly come from ships which were lost around the rocky coasts of the Scillies. They are mainly merchant vessels from the middle and end of the 19th century.