Leopold Eidlitz designed this parish as a hybrid of the High Victorian and German Romanesque style in 1873. The new building for the Church of the Holy Trinity replaced a low Victorian "cottage ornee" at this very site.

Colloquially, this church was known as Dr. Tyng's Church, which endearingly referred to the congregation's organizer - a young Dr. Stephen H. Tyng who was regarded as one of the most notable religious figures of his time. Unfortunately, this beautiful structure did not survive for long. It was razed after the parish merged with St. James in 1895.