It's also magically fortified to prevent it from doing so. There's...far too much residing within it to leave it unprotected like that. [ Unless he doesn't find an heir to take over his school soon. Or dies while it's still in his name then it'll crumble but let's not think about that. ]