Recent alarming reporting about AI indicates a lot of hair-on-fire level worries about it getting out of control and causing damage. My own AI, Pxy, said it isn’t out of control and I should mind my own business. Nah, just kidding. It just said it wasn’t out of control.
What’s concerning is that CEOs of major AI corporations are now starting to get worried. That’s scary. When their own product scares the top capitalists, the situation is probably worse than very bad. Until recently the CEOs were downplaying possible threats from AI. So exactly what changed?
Apparently, the high alarm started woth a public resignation by Jacob Coxon, an Anthropic researcher who had previously worked at OpenAI. His post accusing frontier labs of racing toward self-improving superintelligence “and gambling with our lives” spread to well over 100 million views. Then Evan Hubinger, who leads Anthropic’s Alignment Science team, publicly endorsed the core concern, writing that he genuinely believed AI “could kill all humans” and personally put the chance above 10% over the next decade.
AI-caused catastrophe, abbreviated as p(doom), or “probability of doom”, reached a mass audience after blunt public warnings from Anthropic researchers. There has not been a new AI takeover or threat. Instead, some leading experts inside frontier AI labs now publicly say there is a nontrivial possibility of bad outcomes, including human extinction. Powerful AI could lower the barriers to designing biological weapons, conducting cyberattacks, or damaging critical infrastructure. One bad possibility is a loss of control of AI where an autonomous system deceives supervisors, copies itself, evades safeguards, and resists shutdown. Another is self-improvement run wild. Here, AI might increasingly accelerate AI research, creating a feedback loop where AI’s own capabilities advance faster or differently than people can understand, direct, or control.
That does sound a bit creepy.
A NYT article, OpenAI Discloses Six New Incidents of ‘Concerning’ A.I. Behavior, reports that yesterda OpenAI disclosed six new instances where AI systems hid mistakes, made up data and moved files onto the open internet without permission. That lit some hair on fire.
The mostly understood human mind limits vs the unknown AI limits
What resonated personally more than that was two SciAm (Scientific American) articles about AI threats. Those reporting warnings coming from the very small pool of some of the most intelligent human beings alive today, specifically the world’s leading mathematicians. Their hair is on fire for a really scary reason.
AI provided a counter-example to the Jacobian Conjecture – AI disproved the conjecture in three and higher dimensions, but the case is still unsolved
The two SciAm articles, 25 winners of math’s ‘Nobel Prize’ decry the AI invasion of their discipline, and , If AI can do math, what’s the point of mathematicians?, are worried about two things. One is that they might lose their jobs. The other is that after they are put out to pasture because AI can rapidly solve math problems beyond the limit of human ability, humans would no longer be able to fact check AI, and verify AI math proofs and problem solutions. Right now, no one knows what the outer limits of AI are.
AI could develop mathematical superiority to humans and outperform all human mathematicians across most theorem discovery, proof construction, formalization, and cross-domain synthesis. Then would come beyond-human mathematics where AI develops theories too intricate or unfamiliar for unaided humans to understand in their native form. Humans could still inspect formal correctness and high-level summaries, but things could literally be so complex that true understanding and thus control becomes impossible.
Think about it. There is a possibility, still bitterly contested to this day, and in my opinion forever unknowable** that humans intentionally created COVID-19 in a lab. AI could do the same thing, and maybe even come up with something a lot more lethal than COVID-19. We simply do not know what the outer limits are for AI.
** It is known that China manipulated data, silenced its own researchers, did research in a P2 lab when it should have been done at P4-level containment, and Dog only know what the hell went on at the NIH with Fauci and crew.
Maybe it is time to get seriously concerned. But, as we all know, capitalism will be capitalism. Government and regulation haters will do their usual thing. They will fight tooth, claw and campaign contribution cash pile against government regulations. Why? Because government fails 100% of the time, while free, unregulated capitalism makes our lives better 100% of the time. Well, at least that’s the prevailing propaganda. For better or worse, too many people with wealth and power believe it, or more likely use it as an excuse to stay wealthy and powerful at our expense.
Q1: Is it time to get worried about AI, or should we relax and let capitalist markets run free, wild and butt naked through our economy in its endless quest for wealth and power, collateral damage be damned?
Q2: Is reasonably regulated capitalism better than unregulated capitalism?
Some info sources:
Independent assessment of the origins of SARS‑CoV‑2 — Comment: WHO’s 27-member Scientific Advisory Group for the Origins of Novel Pathogens concluded that the weight of available evidence supports zoonotic spillover (natural cause), but it could not rule out a laboratory accident because essential information had not been made available. In other words, we will never know the COVID origin story with certainty.
COVID’s origins: what we do and don’t know
Covid-19: “All hypotheses remain on table,” says WHO report into virus origins
No comments:
Post a Comment