Not all cases are the same and your approach on creating a space it probably work on all cases, same as it would without but... as you said, adding a French touch might be perceived as a nice detail and a more pleasurable reading.

From my perspective, it also depends on which letter is last near the exclamation point: it does change, having a i! that compensate or a v! that push it forward. The type of font used also make a huge difference. So I'd personally not use the same approach all the times.