TODO: These are just some notes, I haven’t written this and am not sure when I’ll have time to.
Counterarg 5
Bob:
Let’s focus on solving today’s real, tractable problems. When we get closer to systems that could pose existential risks, we’ll have a much better understanding and will be able to take the necessary precautions. By focusing too much on far-off X-risk scenarios, we risk diverting resources, attention, and public trust away from real issues AI presents today, like bias, privacy invasion, automation disrupting jobs, and ensuring the reliability of AI in critical areas like healthcare and safety systems. These are the issues that impact people’s lives right now, and that’s where our attention should be.
Also, maybe current architecture and ASI architecture will be quite different, so research now would be useless.
Also alignment just seems hard. Maybe we should build ASI first and then solve alignment.
Alice:
That’s not even an argument against risk, that’s an argument against some normative claim that I haven’t made yet. Also I’ll note that they are bad arguments.
But just some quick arguments:
Why do you think that aligning ASI is going to be easier than aligning less powerful systems? Why is the fact that we have failed to solve basic alignment problems in simpler models a good argument for just pushing on and saying we’ll deal with alignment later? In math I usually like to solve a simpler problem and then work my way up to a harder problem. “Align a baby AI” seems like a simpler problem than “align an ASI” and aligning a baby AI would be clear progress towards aligning an ASI i dont get why you think that an ASI won’t be built out of MLPs / transformers / RL. Some of these things are fairly old (e.g. RL). I also don’t understand why you think we are so far away from human level AI.
Bob:
Oh ok, I’ll move this to another doc then, thanks.