James Boulter Stokes (1804–1881) was the third son-in-law of Anson Greene Phelps to become a partner in the mercantile business of Phelps, Dodge & Co.
Stokes's parents, Thomas and Elizabeth (Boulter) Stokes, emigrated from England to America in 1798. They settled in an area north of New York on the Hudson River near Sing Sing. Financial difficulties forced them to move to New York, where Thomas started businesses importing fine woollen cloth, selling coal and investing in property. He was a religious man and joined the New York Peace Society and the New York Tract Society, becoming acquainted with Anson Greene Phelps and David Low Dodge. He died in 1832 at which time James and his brother Edward Halesworth Stokes took over the businesses.
In 1833, James Boulter Stokes travelled to England and met his maternal grandfather, James Boulter, for the first time. He found the gentleman to be so objectionable that he immediately removed the name Boulter from his own signature and from that time forward was known simply as James Stokes.
Stokes's younger brother, Josiah, worked for Anson Greene Phelps as a confidential clerk. Josiah was betrothed to Phelps's daughter, Caroline (1812–1881), and the intention was for him to become a partner in the business. On 4 May 1832, Josiah was killed when the warehouse he was working in collapsed due to structural failure. Five years later on 12 April 1837 James Stokes married Caroline.
In 1839 he was in business with the firm of Stokes, Shapter & Walton, importing cloth, and living in England. On his return to New York in 1841 he built a house called Clifton Cottage, in the grounds of Anson Phelps's 35 acre estate on the East River, situated between 29th and 31st street. Later he lived in 37 Madison Avenue and in a house situated between the villages of Old Derby and Ansonia. It was originally an Episcopal rectory, enlarged at the rear.
Stokes provided funding for his father-in-law's Phelps, Dodge & Co. business during the 1837 financial crisis when the banks had suspended payments. In 1847 he was invited to join the business, holding a 15% share in 1853 and 20% in 1858. He became the first president of the Ansonia Brass & Copper Company, the Ansonia Clock Company, and the Ansonia Land & Water Company. Later two of his sons, Anson Phelps Stokes and Thomas Stokes, joined Phelps, Dodge & Co,. Stoke had business interests outside of Phelps, Dodge including the ownership of 38,000 acres of pine land in Michigan.