Remember that Josephus said that ashlars in the Third Wall were no less beautiful than the ashlars of the Temple itself.