Exactly. It none of your business. That is my private medical history. That is between me and my doctor.

It’s not HIPPA but you can say it’s between you and your doctor. Your doctor is constrained by HIPPA. Sue his/her/it’s ass if he/she/it violates it.