This is a view I've always held, as well. Bob Murphy is one Austrian who I think understands the value of mathematics in economic analysis.

Most of the older Austrians conflated "math" with the kind of real analysis that's so common in mainstream theory. I think many of the Austrian critiques of relying on that type of mathematics are fine.

What they miss is that the kind of work Kenneth Arrow did is also highly mathematical. He was making use of powerful deductive tools that apply to the strict paradigm of ordinal preferences that Austrians rightly emphasize.