If you are a genuine individual and are serious:

If it were only facial recognition, I would think it could have been a condition of prosopagnosia (face blindness) setting in. https://infogalactic.com/info/Prosopagnosia

"A cognitive disorder of face perception where the ability to recognize familiar faces, including one's own face (self-recognition), is impaired, while other aspects of visual processing (e.g., object discrimination) and intellectual functioning (e.g., decision making) remain intact."

However, you also noted the sense of impersonation simply by voice over phone and not only when face-to-face is involved. This leads me to think that it may not be (or only be) prosopagnosia, but Capgras delusion or possible a form of schizophrenia. https://infogalactic.com/info/Capgras_delusion

"Capgras delusion is a psychiatric disorder in which a person holds a delusion that a friend, spouse, parent, or other close family member (or pet) has been replaced by an identical impostor." ... "The Capgras delusion is classified as a delusional misidentification syndrome, a class of delusional beliefs that involves the misidentification of people, places, or objects. It can occur in acute, transient, or chronic forms. Cases in which patients hold the belief that time has been "warped" or "substituted" have also been reported." ... "...patients with Capgras delusion may have a "mirror image" or double dissociation of prosopagnosia, in that their conscious ability to recognize faces was intact, but they might have damage to the system that produces the automatic emotional arousal to familiar faces. This might lead to the experience of recognizing someone while feeling something was not "quite right" about them."

Also, if you're using drugs such as ketamine, whether for a separate condition or recreationally, you may want to stop:

"In one isolated case, the Capgras delusion was temporarily induced in a healthy subject by the drug ketamine."

Consider seeking help for it before it gets worse, especially if you develop toward also not recognizing your own reflection and seeing yourself as an impostor or that your mind is in someone else's body or that your body has been changed/replaced. Also, keep in mind that you may also get that sense from those who you would be seeking help from (such as a doctor in a followup visit seeming like an impersonator of the doctor from the first visit despite being the same doctor).

You may want to look into prosopagnosia.

Sounds like you are going schizophrenic.

Better go see a doctor.

Seek help.

I agree with earlier posts. This happened to someone I know. He was sure his co-workers were coming into his apartment, moving things around. Blue cars kept following him. He heard voices, but couldn't tell what they were saying. Was diagnosed with schizophreniaform (sp?), not uncommon among young men late teens into 20s. Was helped with meds. Usually clears up in 6 months or so. Also happened with a friend of my ex, long before the current meds were available. It cleared up around 6 months, wife supported him, has been fine since.

It may just be coincidence, but both men are highly intelligent. Thought it was worth sharing.

If you do seek psychiatric help, be sure to find a doctor who does NOT believe in staying on meds forever. Dr. Peter Breggin has written books on this.

Good luck!

Are you in your mid 20s? That is the prime age range for schizophrenia to manifest and what you describe sounds very much like a form of it to me.

https://en.wikipedia.org/wiki/Prosopagnosia

schizophrenic starts in people in their 20's. Seek help, medicine You need to outthink your own brain

Sorry but I'm as paranoid as it gets and my family are as plain and boring as anyone who could easily get duped into any type of scheme imaginable and this has never happened to me whatsoever. It is more likely your mind playing tricks on you.