Yeah I think if they pulled it off everyone would be like, "What jersey problem?"
The cost of building a stadium there would be absurd, even for royals with hundreds of billions. The entire structure would have to be scrapped and a much bigger and structurally significant one would have to be built. Hard to imagine it would get environmental approval and harder to imagine getting in and out of the stadium for weeknight games.
And also on the fringes of two of Manhattan's wealthiest neighborhoods, that you would need to walk through to get to the subway.
Hard to imagine a less likely patch of open space on the island of Manhattan, other than maybe the sheep meadow at central Park.