Intro to AI Safety, Remastered

I strongly suggest anyone doubtful of the reasoning behind these concerns or asking questions like "well why don't we just do x to solve this?" go check out Robert Miles' YouTube channel[1]. It's quite an approachable intro, often entertaining, doesn't take a long time to get through all the core videos, and very thoroughly answers all the "why not just do x" questions if you go through the whole set. He also does a good job of introducing a lot of the terms used in the field if you then want to go look up papers and get more into the details. One important point he makes is that when you're making a risk assessment you have to both consider the probability of something going wrong and the scope of the potential consequences. When the potential consequences are an existential threat you don't need a high probability to take them seriously. I also happen to think that if you watch all the material he makes a compelling argument that the odds of something going wrong are fairly high unless we start approaching AI research (and particularly safety and ethical concerns) drastically differently than we are now. [1] [this video]

Venkatesh’s argument is so poor it comes close to “not even wrong” territory. He doesn’t really cite any specific people or literature to disagree with, and instead makes only vague references to the “non-sense” of others. The most significant concerns around AI safety are not discussed nor even mentioned leading it to read almost as if he has no expertise whatsoever in this field, and indeed that seems to be the case. No need for an appeal to authority though - just taking a little time to hear the actual concerns and opinions on topic would be welcome. Great intro to the subject here, I wonder what his response would be? [this video]

The basic argument is trivial: it is plausible that future systems achieve superhuman capability; capable systems necessarily have instrumental goals; instrumental goals tend to converge; human preferences are unlikely to be preserved when other goals are heavily selected for unless intentionally preserved; we don't know how to make AI systems encode any complex preference robustly. Robert Miles' videos are among the best presented arguments about specific points in this list, primary on the alignment side rather than the capabilities side, that I have seen for casual introduction. Eg. this one on instrumental convergence: [link] Eg. this introduction to the topic: [this video] He also has the community-led AI Safety FAQ, [link] , which gives brief answers to common questions. If you have specific questions I might be able to point to a more specific argument at a higher level of depth.