pull down to refresh
35 sats \ 11 replies \ @zodthebarbarian 10h \ on: Letter to a Young Person Worrying About AI AI
Dinosaurs didn't see the asteroid coming, and compared to AGI let alone ASI, we're not much smarter. Honestly what is happening is a red alert end times are coming kind of thing. And we have no way of slowing down, lest some other countries like China take the lead. The best and brightest minds in the world are working on it. The biggest companies in the world are going all in on it. And honestly when the next evolution shows up, a species far more capable than us, it'll be like neaderthals taking down a wooly mammoth using spears and planning. The wooly mammoths had no idea what they were in for.
And the most terrifying at all? AGI is not like humans. They are more like a spirit or a ghost, some entirely different entity from us. We have no reasonable expectation that summoning the demon will work out so great for us.
Consider the number one use for AGI: killing your enemy on the battlefield. Military will use AGI as the next iteration of nuclear weapons and we humans aren't going to be able to cope. It's a predicament that will cost us our entire civilization. Call my thinking here doom and gloom, buy I say it's people thinking we have a shot at survival that are living in airy fairy fantasyland. Machines will not be aligned with humans, that would be stupid. And AGI will never be stupid.
Consider how petty humans are. How divided we are. How little we are able to care about each other, let alone work towards the common good of all. Do you think AGI is going to submit to our will? Let alone agree with us? I say enjoy your last few years the way things are because we may very well be the last generations of humans and we think we are way smarter than we really are. The intelligence explosion will show quite clearly that we have become the wooly mammoth while AGI hunts us with spears, planning, and greater numbers. Once the genie is out of the botitle there will be no putting it back in. Humans have never been good at stopping progress once people realize it is possible. Now we as a species are racing towards AGI at breakneck speed blissfully unaware of how epically dangerous it is our to our civilization.
Consider how badly the U.S. did in recognizing the threat nuclear weapons posed. It took a very long time to get scientists onto a secret Manhattan project program, and to stop sharing publicly progress in nuclear physics. It's the same today with AI labs. Spies around the world could steal every secret, every AI weight, because they are locked away in San Francisco at some startup, not in a secure government facility.
IMO we are already locked into WW3, and it will be one fought primarily over who has the decisive advantage in AI. We are seeing the early signs such as NVDA being banned from selling chips in china. That was a first step towards recognizing this extinction level threat. But it is not enough and I'm afraid it may come too late.
Well... AGI doesn't exist...
If this is truly what you believe... what is your personal response? Sounds pretty black pill to me. Not saying you are wrong but views like this always make me curious. Is it helpful? What is the impact of this view of the future on the individual.
reply
Once upon a time nuclear weapons didn't exist either. Scientists thought fission wasn't going to create a chain reaction. But now? We all know what happened. You say AGI doesn't exist but AGI is a line in the sand that we choose. We used to think computers wouldn't beat us in chess. We used to think computers couldn't outdo us on a math competition. There is a clear trend of computers doing anything and everything better than humans.
What is my personal response? First my personal response is to recognize reality. Look up Situational awareness PDF. In there you will see information extrapolating on why AGI is likely to show up as soon as 2027.
Is it helpful? Recognizing reality is always helpful.
What is the impact of this view of the future on the individual? Not deluding oneself into ignoring what is happening. Nuclear weapons were once fantasy but now they are everyday. Chatbots that understand and can think clearly were once fantasy. Self driving cars were once fantasy.
reply
Chatbots do not think. You lost me their. They are probability algorithms. AGI is very loosely defined and its kinda pointless to discuss unless we are talking about the same thing.
We are supposed to be afraid of quantum anyway.
I don't disagree with your thought about knowledge and recognizing reality though. I'm just skeptical about the power of "AI". I do think there are dangers within the scope of that tech though. Even with your examples though... nuclear has been a net good for mankind IMO. It could be used for ill and has been but it hasn't killed us yet.
reply
We can quibble about language but if I ask ChatGPT a question it usually gives me a well thought out answer. Whether or not they think is some philosophical question. The reality is they are quite capable at thinking well, just ask any high school student trying to complete an essay.
AGI is not pointless to discuss; see my original remarks. We are living through the era of intelligence explosion. AGI is one of the most important topics of our lives.
reply
The reason I'm not maybe as concerned as many is because the more I have learned about how LLM's are created, trained, and used effectively the magic of it all has faded away. Its an evolution in computing. I still think it is up to humans what they do with technology. We are the real danger. Not the tech.
reply
Humans are still in the driver seat but with trends the way they are that will not last long. Militaries will use AI to plan their strategies. Drones will be controlled entirely by AI. Making complex decisions that run our world will be done by AI, not humans. Humans as thinkers are quickly being eclipsed by superior machines. Look at the list of things humans do better and you will notice how quickly that list has been shrinking. The trend is clear.
As far as doubting their abilities, i.e. the magic of it all fading away, I would just suggest you consider reading Situational Awareness. Look at the data, extrapolate out a few years and the trend is clear. Just for some low hanging fruit on the subject, just see how quickly ChatGPT has advanced.
reply
well thought out answer.
I know it might seem like a quibble but AI has no intent. Just like the Internet it can be used to magnify good or ill.
reply
Whether or not it has intent is irrelevant. What is your point? That it doesn't "think" like humans? I agree with you there. It doesn't think like us. And that is terrifying.
reply
Why is it terrifying at all? Maybe you should start there?
See, I think making it sound terrifying and magical is actually the worse response we could have. Instead we should be explaining how it functions. Why it fails. Why it gets things wrong so much. What it actually is.
Based on what I have seen it could be a net positive. It also could be a net negative. I don't see it as a humanity ending risk.
reply
Why is it terrifying at all?
Seeing a computer able to outperform humans at all tasks on the horizon is terrifying. No longer being the apex intelligence on this planet is terrifying. Understanding that the number one use for AGI is for military purposes is terrifying. Understanding the alignment problem is terrifying. AGI is not going to be aligned with humanity because we as a society are ill prepared to create an AGI that is humanity aligned. And even if we did it is extremely difficult to prove that what we have created is in fact humanity aligned.
This is a very doom and gloom video that I don't completely agree with but it does discuss some of the main points of Situational Awareness. Will give you a bit of my perspective: https://www.youtube.com/watch?v=k_onqn68GHY
Birds are the new dinosaurs of the deep, says some people! Have you heard of an extinction burst ? Yes!
Charting courses by yesterday's weather!?! No!
A leviathan of our own making is breaching on the horizon—AGI, a storm born from our own signal fires in ten years? Like an earthquake or robot dogs in Singapore? Neither!
But the admiralty is too busy counting rival sails to see the tidal wave forming in the wake.
Debate and discuss who built the ghost ship, but once it's sailing, it answers to no one.
The mammoths never saw the spears. The captains never saw the wave.
And we, proudly at the helm, don't see that we're the FOD.
reply