Castle Island is in the city of Albany, Albany County, New York and has over the past 400 years been referred to as Martin Gerritse's Island, Patroon's Island, Van Rensselaer Island, and since the late 19th century has been referred to as Westerlo Island. Van Rensselaer Island is also the name of a former island opposite Albany, in the city of Rensselaer.
What is now referred to as Westerlo Island was once 5 separate islands; Castle Island/Westerlo Island, Cabbage Island, Bogart Island, Marsh Island, and Beacon Island.
It is claimed around 1540 French fur traders built a stone "castle" or fortified trading post on the island, which they later abandoned. The first definitive European settlement came in 1614 when the Dutch built Fort Nassau under the command of Hendrick Corstiaensen, but in 1618 it was also destroyed by flooding. The island was quickly divided into farms on a lease basis from the Rensselaerswyck patroonship. The first farm on Castle Island was called "Rensselaersburgh" and was established in 1630. The Dongan Charter of 1686, which incorporated Albany as a city, placed the southern boundary as the northern "tip of Marte Gerrite's island", which is what Westerlo Island was referred to at the time.
Castle Island was from 1909 to 1928 the site of the Quentin Roosevelt Field, the precursor to the Albany International Airport. Early aviators such as Glenn Curtiss and Charles Lindbergh landed there. In 1926 Westerlo Island was annexed by the city of Albany from the town of Bethlehem, the other islands were not. With the filling in of the shallow creek and the coastal areas of the Hudson River the three smallest islands; Bogart, Marsh, and Beacon; merged with Cabbage Island by 1929. Island Creek still separated Westerlo Island from the mainland and Cabbage. The Normans Kill (Normanskill Creek) separated Cabbage from the mainland. In the 1930s Island Creek was filled in, connecting Westerlo to the mainland and ending its time as an island, though it is still labeled as Westerlo Island on NYSDOT and USGS Albany quadrangle maps. In 1932 the island was opened as the Port of Albany-Rensselaer, along with 35 acres (140,000 m2) in the city of Rensselaer. The Canadian Pacific Railway's Kenwood Yard and railroad tracks sit on the site of the original creek. A park just north of the Port of Albany is at the site of the former northern terminus of Island Creek with the Hudson River and retains the name Island Creek Park. By the 1950s Cabbage Island too had merged with Westerlo Island, but with the Normans Kill still flowing along its southern and western banks. The land along the Hudson River and north of the Normans Kill within the town of Bethlehem is the site of Cabbage Island.