pull down to refresh
0 sats \ 2 replies \ @optimism 11 Nov \ parent \ on: The lump of cognition fallacy - Andy Masley AI
why? you can easily have such a conversation in ~bitdevs? just won't be instant answered, but mistakes will be caught when it's done in public.
That 24/7 access to a data terminal with a conversational interface is indeed what current-gen AI excels at.
Yes, it's still better to get direct human feedback, but that isn't available all the time, as the time and attention of the humans are much more valuable
reply
As long as we remember that:
- @ihatevake is, per indirect evidence at SNL tabconf, human too, so what about the time wasted on a bad answer?
- Chatbots are trained to be obedient and have zero accountability. A chatbot can get away with murder (literally.) This means that the answer you get is extremely low value.
I have an open item triggered by something Justin said about static device keys in his podcast interview the other day and I need to write something about it. Based on this conversation we're having here, and as a fun ad-hoc experiment, I asked both chatgpt and grok something that I have decades of experience with:
- Why is it a bad idea to use static device keys?
- Why is it a good idea to use static device keys?
In isolated chat contexts so that the trickery is all mine.
The short result is that Grok did, after a whole lot of slop, mention on the first question why: because it violates the assertion that cryptography is soft security and every key will eventually be compromised, even though it forgot to mention that every key thus must expire. This is a make-it-or-break-it characteristic of applied cryptography: a key is a consumable, just like a safe or a fence. Everything else, including both responses from ChatGPT, was slop, and dangerous "advice". ChatGPT did mention that there are "some" scenarios where it's not a good idea on the positive question, but the argument wasn't coherent, because it focused on process difficulty, not intrinsic weakness.
TLDR; it would be extreme Gell-Mann amnesia for me to tell anyone that the bot provides valuable information. The output is low-value.
reply