Dude, I used to live in that area of NYC. The rents that could be collected if the building was torn down and made into apartments would be astronomical. (Not withstanding whatever happens to NYC's real estate market post-Corona.) The consulates and delegations would no longer need the expensive brownstones they occupy and those could also be put on the market. There are so many ways NYC could better leverage the space, even if it wasn't to allow for residential housing.