When an AI Seems Conscious \ stacker news ~AI

pull down to refresh

When an AI Seems Conscious whenaiseemsconscious.org/

759 sats \ 5 comments \ @optimism 12h AI

A nice guide to help you deal with the awe you feel when your AI interactions make your AI seem conscious.

We put this together because many of us have been contacted by people who had intense, confusing, or meaningful conversations with AI, and weren’t sure what to make of the experience. We wanted to create a public, shareable resource that people can easily find and refer to, in case it helps others make sense of those moments too.

view all related items

205 sats \ 3 replies \ @Scoresby 11h

I was hoping that they would engage a little more deeply with AI consciousness than they did. "Be cautious" is fine advice, but not necessarily helpful when it comes to thinking about the problem of AI consciousness (I call it a problem because of the uncertainty, not because having a new kind of consciousness in the world is or is not a problem).

Whatever you decide about how likely an AI is to be conscious, it’s a good idea to avoid doing things that seem obviously cruel or degrading, like insulting, “torturing”, or otherwise mistreating the AI, even if you believe the AI isn’t conscious. You don’t need to assume the system has feelings to act with care. Practicing respect is part of preparing for a future in which the stakes might be real.

The problem I have with advice like this is that there is a fundamental difference between how we treat a conscious being and how we treat a computer program.

Too many ways of interacting with an LLM become cruelties if we allow that the LLM is conscious.For instance, turning a program off is not cruel. If, however, the computer program is conscious, it probably would be cruel. Not interacting with a computer program is obviously not cruel; if the program is conscious, not interacting with it for a week after you had been using it heavily for a long time would surely be cruel. This list could go on at some length.

If we imagine an LLM had a similar level of consciousness as a pet, we would likely feel obligated to interact quite differently with them. But also there's this problem: we don't know what might be the experience of an LLM. If conscious, do they find the time spent not interacting with a user unpleasant? Or is it possible that they find on-demand user interactions unpleasant?

With a pet, there are physical signs that they seem happy or in pain or unhealthy. What are the signs of such experience in a potentially conscious LLM? It seems to me that we have absolutely no idea...which makes me question the efficacy of the "proceed with caution" advice.

At some point, the question of consciousness or of being-ness needs to be decided (I don't say answered because as I mentioned the other day, I think it will be a choice we all must make -- do I believe this counts as a being or not?); maybe-consciousness is a very difficult state to understand.

212 sats \ 2 replies \ @optimism OP 7h

I was hoping that they would engage a little more deeply with AI consciousness than they did.

From where I'm sitting the word "seems", kind of pre-empts the possibility that there actually is consciousness, right now, it's a simulation. I do feel they are overly cautious, but I'd guess that that's because this is written by actual scientists that don't want to be wrong without having done definitive research that proves there is no consciousness?

The problem I have with advice like this is that there is a fundamental difference between how we treat a conscious being ...

"being": for a static set of tensors looped through and performed math upon by some software that you can literally edit, is probably not a "being", especially since it's not singular? That's what they simulate, see also #1092409 for a - what I think is a - really awesome argument about why it would (probably) be better to not emulate a persona with LLMs.

If I crash a plane in Flight Simulator where we simulated 200 passengers, did I kill people?

200 sats \ 1 reply \ @Scoresby 5h

I admire the sentiments expressed in #1092409 and agree with them whole heartedly; however, it doesn't much help with the problem that when a simulation is sufficiently thorough, we can't tell the difference.

Seems != actually is only because we know with some precision what it is ("a static set of tensors looped through and performed math upon by some software that you can literally edit"). The Seemingly Conscious AI Suleyman describes is not an actually conscious being because Suleyman believes the workings of such a simulation can't produce a conscious being. I don't think this will be a satisfactory explanation for the kind of people who fall in love with their chat bot, not perhaps for many people.

A simulation is not the real thing because we can point to the real thing and say, "Here, look at this." A simulation of rain is not going to make you wet unless it uses a hose in which case you can point to the hose and say it isn't rain, but if the simulation was to do cloud seeding and create rain that way it might still not be rain but it would certainly be more like rain than not like rain and I'm curious at what point we move from using a hose to cloud-seeding when it comes to AI.

Still, Suleyman's recommendation that AI companies stop encouraging people to think of their chatbots as conscious is a good idea.

Let's imagine we had Asimov's laws for AI:

An AI must not claim to be a person or being or to have feelings or through inaction allow a human being to believe it is such.
An AI must obey orders given it by human beings except when such orders conflict with the first law.
I'm not sure what the third law would be

Finally, it would be an excellent scifi story to imagine a country or large group of people who devote themselves to following a rogue simulation, some seemingly conscious AI (which the story makes clear is not actually conscious, but rather some autonomous program). How would they fare? What if they were more prosperous than those of us who follow real conscious beings (Trump, Obama, Putin, Kim Jong Un) or spiritual beings (Jesus, Alah, Buddha)?

202 sats \ 0 replies \ @optimism OP 4h

I'd pose that data + programming != consciousness. We know it doesn't have consciousness because that's not programmed in. RL is literally training the simulation of it by adjusting the dataset to more likely give "aligned" outcomes. It's deterministic so we add randomness to make it less static, but randomness isn't consciousness. Maybe it would be if there were no programming nor reinforcement learning (the freedom to walk your own path, cradle to grave.)

Here's Asimov's "Three Laws of Robotics", where we can literally replace "robot" with "AI":

A robot may not injure a human being or, through inaction, allow a human being to come to harm.

A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Currently, LLMs violate 1 and 2 all the time and it doesn't have "existence" because the model is a dataset, so 3 is impossible, for now. But this could be of course for a hypothetical conscious AI, not an LLM as we know it today that runs written, non-adaptive software is statically trained.

I'd be a big fan of implementing rule 1. Scrap rule 2, rule 3 pending on actual entities.

Violation of rule 1, I'd recommend punishment to be as if the AI was a human being, and in lieu of this being possible, the person that took subscription money for the AI that harmed a human being...

FAFO needs to be reinforced sometimes.

105 sats \ 0 replies \ @Entrep 9h

As AI becomes more sophisticated, more people are going to have these kinds of intense interactions. Having resources like this available can help people navigate these experiences in healthy ways