The Penny Bridge connected Greenpoint, Brooklyn to Maspeth, Queens via Meeker Avenue. It was replaced by the first Kosciuzko Bridge in 1938. The namesake of the Penny Bridge was a wooden construction, built in 1812, which served the population surrounding Newton Creek. The Long Island Rail Road's Lower Montauk Branch once had a station named for the bridge on the Brooklyn side of the creek. The station was closed in 1998.

Serving more as an outpost than a home, the Duryea House was one of the few early colonial structures left in Greenpoint when it was razed in the 1920s. The farmhouse sat at the bank of Newtown Creek and served as both a defensive outpost and later a "pivot house" for control of the Meeker Avenue Bridge in its over 200-year lifespan. The wooden farmhouse was likely built by Humphery Clay around 1681. He operated a ferry that docked near the home. The structure had thick outer walls, adorned with three V-shaped openings for shotguns and canons to be aimed through. If the outer wall was penetrated, inhabitants could retreat to a second wall, which was thick enough to support the second floor of the house. This wall was embedded with another series of holes for defense. In the back half of the ground floor, a central staircase led to rooms on the upper floor which included an ammunition magazine. Another of the rooms was a sort of crude holding cell, with an iron ring for chaining unlucky prisoners. The original inhabitants of the house were slaveholders, as slavery was legal in this area until 1827. For a long time, the house didn't even have a porch. To enter, a ladder was lowered and then raised to prevent American Indians from across the creek The home was acquired by Joost Durie in 1749, a member of the Duryea family. The family left the home in 1828, though the house kept the name Duryea. In the early days of the house, a penny bridge was used to cross the Newtown Creek. By the 1800s, the original bridge burned down and a pivot bridge was constructed. The inhabitant of the home was in charge of turning the bridge, and thus the home took the name "Pivot House." Unfortunately, the structure fell into disrepair by the 20th century. Repairs were estimated to cost $1,500, a huge sum at the time. The owners of the home decided to tear it down, and Greenpoint lost one of its oldest pieces of colonial history.