I feel a little bit unsure about your reasoning. You made the same point a few times: "Computers can't think because they don’t have the hardware for thinking." and based it on the example of the man who lost his feelings and therefore agency and the idea that computers are only able to copy based on some previously provided data.
The fact that somebody lost his agency by loosing his feelings speaks very little of how an AI can work, since as far as I know current AI are not that close to how our brains work . AI seem to be also getting quite creative with games: https://www.youtube.com/watch?v=Lu56xVlZ40M . I think that not having million years of evolution and little sociological context is an advantage for AI since it's not confined to conventional solutions.
I get the argument in general: AI is definitely going to be different than humans. It might not grasp the concepts of art (or a lot of other concepts we use daily) and even if it does, after learning on millions of examples, it will feel superficial for us.
My fundamental point is: We shouldn't compare ourselves to AIs and AIs to us too much. Which is exactly what you say too. And from that follows, in my opinion: We shouldn't be too afraid that human-like AIs will start to compete with us and win.
Of course there are risks in AI. There are risks in every kind of production. But I remain skeptical of the idea that AIs will go rogue and follow their own agendas. I find it much more likely that many people will die because an AI turned out to be too stupid than because it turned out to be too smart. Like the Tesla car that drove into a truck that was too similar to the sky, but worse.
I think Scott showed so clearly both What is a thought? And What is art? A stained glass window picturing Alexandra Elbakyan and her raven. That really is a thought. Maybe no human being has thought of that combination before (I can readily admit that I haven't). Art is supposed to be based on such novel thoughts that can be made into pictures. Paintings without such new concepts behind them are not considered art, but just paintings. And that is exactly what an AI can do: It can't do art at all because it can't think or even handle new, unusual thoughts. But as far as I know, it can do paintings fairly well.
The fear I have stem exactly from the fact that AI will not have the million years of evolution and the cultural context :) I'm not scared it will go rogue, I'm scared it will follow our orders, very exactly and effectively, and in a way that wasn't what we had in mind - just like in the game in the video.
The video was really interesting. And I basically agree with you - never fully trust a computer, unless you really know what it is doing. But don't trust humans too much either, because they tend to have their own agendas. In general, I think people trust other people more than they should and are more afraid of computers than they should be. But only in relative terms - both are dangerous in their own ways.
What makes me anti anti AI is articles like this one: https://astralcodexten.substack.com/p/deceptively-aligned-mesa-optimizers I just completely fail to be afraid of that strawberry picker. I guess I should be able to overlook details like a strawberry picker needs to be gentle, what is a man with a red nose doing close to an agricultural machine anyway?, why would a strawberry picker have a function for throwing?, why doesn't anyone care that the strawberries are thrown away instead of picked and sold? But I can't. The whole scenario is just too unrealistic to take seriously. If someone has come up with a just slightly more realistic scenario for disaster, I'm eager to hear about it and maybe change my mind.
I feel a little bit unsure about your reasoning. You made the same point a few times: "Computers can't think because they don’t have the hardware for thinking." and based it on the example of the man who lost his feelings and therefore agency and the idea that computers are only able to copy based on some previously provided data.
The fact that somebody lost his agency by loosing his feelings speaks very little of how an AI can work, since as far as I know current AI are not that close to how our brains work . AI seem to be also getting quite creative with games: https://www.youtube.com/watch?v=Lu56xVlZ40M . I think that not having million years of evolution and little sociological context is an advantage for AI since it's not confined to conventional solutions.
I get the argument in general: AI is definitely going to be different than humans. It might not grasp the concepts of art (or a lot of other concepts we use daily) and even if it does, after learning on millions of examples, it will feel superficial for us.
My fundamental point is: We shouldn't compare ourselves to AIs and AIs to us too much. Which is exactly what you say too. And from that follows, in my opinion: We shouldn't be too afraid that human-like AIs will start to compete with us and win.
Of course there are risks in AI. There are risks in every kind of production. But I remain skeptical of the idea that AIs will go rogue and follow their own agendas. I find it much more likely that many people will die because an AI turned out to be too stupid than because it turned out to be too smart. Like the Tesla car that drove into a truck that was too similar to the sky, but worse.
I think it's great you bring up the concept of art, because it was on my mind too since the ACX post on the subject.https://astralcodexten.substack.com/p/a-guide-to-asking-robots-to-design
I think Scott showed so clearly both What is a thought? And What is art? A stained glass window picturing Alexandra Elbakyan and her raven. That really is a thought. Maybe no human being has thought of that combination before (I can readily admit that I haven't). Art is supposed to be based on such novel thoughts that can be made into pictures. Paintings without such new concepts behind them are not considered art, but just paintings. And that is exactly what an AI can do: It can't do art at all because it can't think or even handle new, unusual thoughts. But as far as I know, it can do paintings fairly well.
The fear I have stem exactly from the fact that AI will not have the million years of evolution and the cultural context :) I'm not scared it will go rogue, I'm scared it will follow our orders, very exactly and effectively, and in a way that wasn't what we had in mind - just like in the game in the video.
The video was really interesting. And I basically agree with you - never fully trust a computer, unless you really know what it is doing. But don't trust humans too much either, because they tend to have their own agendas. In general, I think people trust other people more than they should and are more afraid of computers than they should be. But only in relative terms - both are dangerous in their own ways.
What makes me anti anti AI is articles like this one: https://astralcodexten.substack.com/p/deceptively-aligned-mesa-optimizers I just completely fail to be afraid of that strawberry picker. I guess I should be able to overlook details like a strawberry picker needs to be gentle, what is a man with a red nose doing close to an agricultural machine anyway?, why would a strawberry picker have a function for throwing?, why doesn't anyone care that the strawberries are thrown away instead of picked and sold? But I can't. The whole scenario is just too unrealistic to take seriously. If someone has come up with a just slightly more realistic scenario for disaster, I'm eager to hear about it and maybe change my mind.