I agree that there is some type of a security risk. Maybe not so much now but in the future for sure as BTC continues to appreciate in fiat terms. I have gone from wearing a hat with a bitcoin logo on it to wearing the honey badger hat from coinkite. I feel this is subtle enough that people in the know will get the reference but normies will just see a honey badger.
There are a lot of things that we do all the time that could mark as well off though too. Driving a nice car, shopping at certain places, wearing a fancy ring, wearing a watch, etc, etc. The amount of people doing those thins outweighs the small number of people with bitcoin paraphernalia.