The small white farmhouse now known as Poe's Cottage was built circa 1812. The structure once occupied a plot across the street from its current location in Poe Park (once a large apple orchard). In 1846 poet Edgar Allan Poe and his wife Virginia began renting the cottage for $100 a year. Virginia, who suffered from tuberculous, died in 1846 and Poe lived in the cottage, at least part of the time, until his death two years later.

Around the turn of the 20th century, the home was in danger of demolition because of the proposed widening of Kingsbridge Road. A local literary group petitioned for the house to be moved across the street and to have a park created around the structure. Poe Park opened in 1902 and was a bucolic area of winding paths, trees, and flowerbeds. In 1913 the home was moved once more (only a few hundred feet) to its current position and has been in the care and custody of The Bronx County Historical Society since 1975.