The Leake and Watts Orphan House was erected in 1843, and the massive Greek Revival structure is an unlikely survivor in an ever-changing city. Designed by architect Ithiel Town as a home for orphaned children with the funds from a wealthy New York lawyer, the building was the largest edifice in Morningside Heights until the construction of the neighboring St. John the Divine began in 1892. (The Cathedral is still officially unfinished more than 11 decades later.)

The orphanage moved to Yonkers when the church purchased the building in 1887, and the asylum stands on land intended for the south transept of the cathedral. (See the above parenthetical for the reason this has not necessitated the demolition of the building.) The architectural writer, David Dunlap, described the scene as this:

"The situation has yielded a happy architectural incongruity: two ample porticos with broad triangular pediments on giant Ionic colonnades, nestled at the base of an unfinished Romanesque-Gothic cathedral. From some vantages on the verdant grounds, it looks as if an ancient Greek temple stumbled into some medieval village."

While the building was not demolished for construction, it was largely ignored by the church and might have been taken down due to safety concerns eventually. But in the early 2000s, the Cathedral reversed it hands-off plan for the asylum. With a massive restoration effort completed in 2006 and the building renamed the Town Building, for its architect, the once decaying structure has become home of the Cathedral’s Textile Conservation Lab, and the Cathedral’s social service and neighborhood outreach programming. A coop in the nearby parking lot is also the home base for the Cathedral's three peacocks, Phil, Jim, and Harry.