In fact, there's been long periods where fees were high enough that some mining pools weren't mining low fee txs at all, making it downright trivial to double spend. Eg this write-up was done entirely with full-rbf during one of those periods: https://petertodd.org/2016/are-wallets-ready-for-rbf

Worse, if somone wants to exploit zeroconf one way to do it is to attack the p2p network and miners, eg with sybil attacks. We really don't want to give people incentives to do this.