My wife's grandmother was the victim of an AI impersonation scam. One of my wife's cousins -- a different grandchild to her grandmother -- had a youtube channel. Nothing big, really just him streaming playing video games. Had maybe 30-40 subscribers....but he spoke enough during the videos that they were able to use his voice for training.

Somehow, they managed to call his grandmother and using a fake script, he said "Grandma I've been arrested they say for trying to steal a car, I didn't do it, but they say I need to pay $2500 for bail to get out....can you please send the money?"

Being a smart older woman, she knew somethings didn't match up (for one he has vision problems and can't drive, secondly he is kinda autistic and being involved in a car theft is last thing he would do)....so she asked for a call back number and called her son (the boys uncle) to come over and they called together.

The boys uncle quizzed him and they soon found out he couldn't answer basic questions about the family, like...oh you called your dad and he couldn't help?? (his dad died when he was an infant), etc.

So no doubt this scam was perpetrated by low-level amateurs who didn't really do any background checking. However, it highlights how difficult it would be to guard against this if you were dealing with professionals who put the time into it.