Royal fam wouldn't let that happen. Harry has to be aware of how Brit nobles secretly talk shit about his marriage; prolly an unspoken reason why he walked away.