Here are 10 takeaways that leaped out in Aschenbrenner's 50,000-word, five-chapter,
165-page paper, "Situational Awareness: The Decade Ahead":
1. "Trust the trendlines ... The trendlines are intense, and they were right."
- "The magic of deep learning is that it just works — and the trendlines have been astonishingly consistent, despite naysayers at every turn."
2. "Over and over again, year after year, skeptics have claimed 'deep learning won't be able to do X' and have been quickly proven wrong."
- "If there's one lesson we've learned from the past decade of AI, it's that you should never bet against deep learning."
- "We're literally running out of benchmarks."
3. It's "strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer."
4. "By 2027, rather than a chatbot, you're going to have something that looks more like an agent, like a coworker."
5. The data wall: "There is a potentially important source of variance for all of this: we're running out of internet data. That could mean that, very soon, the naive approach to pretraining larger language models on more scraped data could start hitting serious bottlenecks."
6. "AI progress won't stop at human-level … We would rapidly go from human-level to vastly superhuman AI systems."
- Superintelligence, coming in 2030 A.D.?
7. AI products are likely to become "the biggest revenue driver for America's largest corporations, and by far their biggest area of growth. Forecasts of overall revenue growth for these companies would skyrocket."
- "Stock markets would follow; we might see our first $10T company soon thereafter. Big tech at this point would be willing to go all out, each investing many hundreds of billions (at least) into further AI scaleout. We probably [will] see our first many-hundred-billion-dollar corporate bond sale."
8. "Our failure today to erect sufficient barriers around research on artificial general intelligence "will be irreversible soon: in the next 12-24 months, we will leak key AGI breakthroughs to the [Chinese Communist Party]. It will be the national security establishment's single greatest regret before the decade is out."
9. Superintelligence "will be the United States' most important national defense project."
10. There's "no crack team coming to handle this. ... Right now, there's perhaps a few hundred people in the world who realize what's about to hit us, who understand just how crazy things are about to get, who have situational awareness."
Reality check: Aschenbrenner, with roots in the
effective altrusim movement, is an
AI investor. So he's not a disinterested party.