LLM training produces a black box that can only be tested through prompts and output token analysis. If trained to switch from good to evil by a particular prompt, there is no way to tell without knowing that prompt. Other similar problems happen when an LLM learns to recognize a test regime and optimizes for that, rather than the real task it's intended for — Volkswagening¹ — or if it just decides to be deceptive. Bad enough. Deliberate training to mislead and disrupt is the most insidious.

The obvious way to uncover such things is to trigger the deviancy. Attempts to guess the trigger prompt are as successful as you might guess. It's worse than brute forcing passwords. You can't do it very fast, there's no way to know quickly whether you've triggered. And there may be nothing there in any case.

A more adversarial approach is to guess what the environment will be when the trigger is issued. Miles gives the example of an AI code generator that is primed to go rogue when used in deployment. By persuading the system that it's in the target environment without putting in the explicit prompt, it may decide to switch behaviors anyway. This doesn't work, and runs the risk that the LLM will become more adept at deception in response.