In 1851, two brothers (sons of early 19th-century medical innovator Dr. Valentine Seaman) purchased a hillside property near the top of Manhattan and began to erect a substantial summer home constructed with locally-quarried white marble.