If you're worried about this, the first thing you need to do is to run your own node. If your hardware wallet is querying a third-party node, you have to assume that they have your xPub key and already know all your addresses.

You don't have to trust any third-party nodes when you run your own node. You can verify everything yourself.

Additionally, I would suggest that you use a wallet that gives you manual control of your UTXOs as well as labeling. I personally use Sparrow wallet because I believe it is the most feature-rich wallet in the world.

I actually wrote an article on how to use bitcoin anonymously here: https://www.whatisbitcoin.com/privacy/how-to-use-bitcoin-anonymously

Hope you find something useful in here and am happy to help you with any questions you may have.