pull down to refresh

I truly think there is something revolutionary about where we are with AI technology.
Agree
Then those neural nets can be "trained" quite generically using a system of carrots and sticks. I believe the computers will be able to optimize against those carrots and sticks even without new human generated corpi.
My point on "needing continual human effort" is that there is still a human (a team in fact) deciding deciding all that. The resulting AI are not deciding "hmmm....today I'm going to go learn how nuclear engineering works". They have no consciousness. No aesthetics. No desires.
I know you were not suggesting they did have any of those things, I'm simply making clear that without constant human direction, they are incapable of doing anything. They are a screwdriver or a buzzsaw.
The danger is that a naive public will believe that it has agency and ascribe to it all manner to deference.