The Madison Avenue Baptist Church once stood on the corner of 31st Street and Madison Avenue, just next door to this small structure. The building was replaced in 1930 with a 17-story residential hotel with space for the church in the first four stories. The previous church had been constructed in 1858 in the Romanesque Revival style, which informed the design of the much later Parish House, which itself replaced a chapel on this site. The Parish House retains an old sign with its previous namesake. On a much grander scale, just down the block, is an Eclectic loft building at number 36, which originally housed garment industry firms. The structure features double-height fluted columns at the first and second stories and the top two stories, as well as ornamental brickwork and carved stone accents.