If my hardware wallet is physically taken from me, I consider it insecure right there. It does not matter if its susceptible to attack. It is a good idea to mitigate trust on the device itself too and use a secondary pass phrase with the seed phrase (also known as the 25th word or 2factor seed phrase) containing your real funds. That way if your device gets stolen and seed phrase compromised you can restore the seed phrase to a new device, enter your passphrase to access the hidden funds and then move everything to a new master seed phrase.

In the event of physical attack or theft I am only trying to buy enough time so the attacker thinks they got something and to transfer my real funds.

Because of that I don't really concern myself with physical CVE's. Theft of my device is compromise of the device.

I advise everyone to use 2 factor seed phrases. It is the most secure way you can possibly store your funds. Even prevents the not impossible but impossibly rare chance someone generates your seed randomly and steals your funds. https://en.bitcoin.it/wiki/Seed_phrase#Two-factor_seed_phrases