21 sats \ 6 replies \ @grayruby 10 May \ on: Has generative AI already peaked? AI
Hard to imagine it has peaked.
Agree, but the argument is that the idea of having one dataset and model for everything is not infinitely scalable. There are diminishing returns there. Perhaps the title of their video is a bit disingenuous.
reply
I agree with this. AI has come to stay and it will continue getting better (at least to some extent). But I also think it is overhyped and I have doubts if we will ever see artificial general intelligence (AGI). Chat GPT is good at guessing the next word in a sentence, but is it really intelligent?
reply
Surely we can admit it displays multiple measures of intelligence and reasoning. I like the example Ilya Sutskever discusses with Jensen Huang- if you train an LLM on text and then feed it a mystery novel and at the end ask it "who is the murderer?" and it gets the answer right, is that not demonstrating that the model is fundamentally learning not just about guessing words but also developing an understanding and internal model of the outside world?
I think we also forget how stupid the bottom 25% of humans are... if ChatGPT is smarter than 90% of people on 90% of things, surely we can grant that it has intelligence even if it is not AGI.
reply
That's a good comparison, but the essential question is - will these models ever be so good that will be able to do that? If they are at some point then the question whether the model is intelligent, becomes irrelevant (Edsgar Dijkstra made the comparison that the question of whether machines could
think was “about as relevant as the question of whether Submarines Can Swim"). However I still have some doubts.