Manhattan's oldest surviving house, the 1765 Morris-Jumel Mansion, is today located at West 160th Street and St. Nicholas Avenue. Perched upon Mount Morris, Harlem Heights' highest elevation, the house is now set within a ten-acre city park and is flanked by Victorian row houses and apartment buildings of a later date. Originally, it dominated the open countryside, occupying some 130 acres between the Harlem and Hudson Rivers.

Mary Phillipse Morris and Colonel Roger Morris moved into the mansion the year after it was completed in 1765. Public agitation against the English Crown soon forced Loyalist Morris, a member of the King's Council that governed New York, to flee to England with his family. In 1776, General George Washington established his headquarters there; after American forces evacuated Manhattan, Hessian soldiers fighting for England made it their camp. In 1779, the property was confiscated by the State of New York. It served for some years as an inn. The most imposing of houses built by English colonists in Harlem, the Morris Mansion also is among the earliest examples in the United States of a temple-fronted house with a giant order of columns. During the nineteenth century, the mansion was wrongly regarded as a rare example of southern architecture in the North. In fact, both the antebellum plantation house and the Morris Mansion were inspired by work of sixteenth-century Venetian architect Andrea Palladio, popularized and modified by British architects. In April 1810, French-born wine merchant Stephen Jumel and his wife, Eliza Bowen Jumel, purchased the mansion and the surrounding thirty-six acres. Along with many of his compatriots, Jumel had fled the Haitian revolution and settled in New York City. The Jumels renovated the mansion, generally preserving its interior. New additions included the front door, with a fanlight and leaded-glass sidelights inset with stained glass, a pioneering innovation.

Madam Jumel died in 1865, leaving Harlem's finest house as a monument not only to its previous inhabitants but also to the entire eventful past and transforming fortunes of the area. Her heirs soon began a protracted fight over her estate that ended with a marathon auction. The house came into the possession of her great-nephew. In 1903, the City of New York purchased the property. The following year, it opened as a house museum, now administered by Morris-Jumel Mansion, Inc