Agreed. There will be different wallets for different things. So long as people have great open source options to fulfill the most basic, yet most important functionality of a wallet, we're good.
Over time, there will be increasingly more 'wallets' that are custodial or pseudo custodial (although things like greenlight, sensei sound amazing as a nice in-between), but these wallets will provide features catering to a specific group of people, with different tradeoffs. We're already starting to see it play out, too.
I guess when someone says, 'we need to build a better wallet', they're generally talking about what THEY want to see in the wallet. Build it and let the market decide ;)