I think the issue with Chainalysis' hesitance to show any of their software or code to the court isn't that it would expose them as being ineffective (which I doubt), but rather that it would destroy their business model by exposing them as the same as any other working blockchain analysis tool with a different coat of paint.
Chainalysis works on a business model where they make a program that can analyse and visualise the blockchain based on address, txid, or other metadata and then complicate it to make a business model of selling you 'advanced training' on how to use their software and then sell you the software AGAIN once you know how to use it. Chainalysis sells certs for investigators or possible law enforcement on the basics of Bitcoin, cryptocurrency and tracing a transaction for £600/$750 - and that is only for the lowest tier!
Do you know what Bitcoin is? Do you know what the differences between Bitcoin, Ethereum and Monero are? Do you know what a blockchain explorer is, and are able to get an open-source blockchain visualisation tool off of GitHub? Congratulations! You basically have a Chainalysis Cryptocurrency Fundamentals Certification1 without needing to buy a training course that costs more than a computer.
Chainalysis do not sell anything different than what a blockchain explorer application could do from what I can tell, and they really don't want you to believe that because that destroys the whole point of the product's business model. They want you to believe they do something special in comparison to organisations like Ciphertrace or partners like Cellebrite and Maltego. And, they can get away with it because law enforcement digital forensic investigators often are not technical, they know how to use forensics software but lack knowledge on doing the process by hand. Chainalysis sell a a blockchain explorer with unreliable, unattributable 'behavioural analysis' or other buzzwords that can't, nor ever been, used in a law enforcement scenario because it is not forensically viable evidence, having a program saying "I guess" is not evidence.
Commercial blockchain analysis software alone has never been enough to convict a person of a crime, because they need forensically sound evidence that the cryptocurrencies of the suspect belong to the person. They are basically tools made to speed up the investigation by babying the cops into doing it all for them without knowledge or with their 'advanced' training programs.
Other digital forensics tools also rely on this model of being needlessly complicated, including some I have definitely used before. If I wasn't given a bachelor's degree on how to use EnCase (including training from Opentext, their vendor) I'd have definitely been stuck. Cellebrite is also crazy expensive, but if you ever watch a video of the software its basically just pressing buttons on a tablet UI and having it do it all for you.
Chainalysis has always had an unhealthy relationship with other blockchain analysis firms, like Ciphertrace who are claiming their results are unreliable as there's no study or reproduction of their results. Chainalysis software is a walled garden with no reproducibility or transparency or research 2. I challenge you to search "Chainalysis" in Google Scholar and find an academic article that contains a screenshot of Chainalysis software or a report generated by their software for the purpose of the academic research.
They either don't want to show their secret sauce so it can be copied or they don't want people to shoot holes in their model. People taking shots at it chain analysis be it narratives, be it methodology, be it on-chain techniques all form part of improving users privacy, keep fighting the good fight.
In my opinion this is big controversy, the lack of audits, yet the software is allegedly being used to sentence people for crimes. Its a big deal and im glad some light is being shed on it.
What's amazing about all of this is that the government would rely on such a closed-source and unvetted software as Chainanalysis in the first place.
It reminds me of all those fake and biased "fact checkers", and "hate group" and "disinformation" lists that the government relies on to censor people.
Just goes to show: the way to get a government contract isn't to provide quality service, it's to
Footnotes