The difference is that you don't need separate infra just to generate new addresses for every user, but of course you still need a back-end to sync your wallet, just like any other Bitcoin wallet.

In the future this will likely just be an extension of Fulcrum/Electrum as there are already forks that add Silent Payment sync to these in a privacy-preserving way, meaning a Silent Payments user could sync using a public remote node without sacrificing privacy, thus requiring no infra.