Vincent J. Cannato’s American Passage: The History of Ellis Island aims to cover the whole recorded history of the island. The book starts with the history of the island long before anyone thought of it as a way station for immigrants.
The Dutch named in Little Oyster Island, as it sat among the oyster beds near the Jersey shore. He notes that the ownership of the island is unclear between the late 17th century and 1785, when an advertisement appeared trying to sell the island along with a couple of lots in Manhattan and “a few barrels of excellent shad and herrings…” (Kindle location, 482).
By the time of the War of 1812, the island had been acquired by the War Department as part of the defense of New York harbor.