pull down to refresh

What happens when you pair a sycophantic bot with another sycophantic bot? Will you get a sycophantic singularity?

Yes. This can happen in review, and does, rather often. For example, Claude's protection against prompt injection will cause role=user input to weigh heavier than role=tool, so it will be much harder for the bot to argue against user input if tool input proves the user wrong.

To work around this I often provide user input indirectly through a tool call, so that issues discovered from code reading tool calls don't weigh much lower than any conflicting information provided. This also has consequences - sometimes the request isn't fully honored if the tool call description deviates from the initial ask (i.e. you ask a bot to read and answer a ticket, the ticket asks for implementation, then about half of the time you'll only get the answer and not the implementation.)

So multiple layers of redirection is both a solution, and a problem, I think.

reply