I think that would work to communicate stuff, and w/ the progress of the deep learning models the synthesis would probably be good; but it would be hard to synchronize with multiple people, I would think. I really want it to capture the warmth and interaction of real conversation, but now I'm kind of curious how your idea would feel. Will update if I try it.