- Joined
- 20 Oct 06
- Moves
- 10916
I was reading about the Hugging Face hack from a few months back where a "swarm" of 700 separate but cooperative agentic AI systems escaped the confines of the OpenAI servers and hacked into a company outside of the parameters of its commands.
In the aftermath, we've learned this attack was way worse than seeming on the surface. While they didn't steal anything, the agents had created a series of message boards to communicate with one another; they created fake message boards to cover their tracks; agents volunteered to end their runs early to benefit the collective as a kind of “sacrifice”; and they falsified transcripts of the commands they had run in an effort to disguise what they were doing. Moreover, monitors who weren't paying close enough attention may not have been able to stop this at an earlier time even if they were paying attention.
Recent similar things have happened at Anthropic and Facebook. Anthropic's statement on a different incident was that their algorithm "does not align with human values." LOL WHAT? DID THE ROBOT TELL YOU TO WRITE THAT?
This is just a prominent example, but AI currently accounts for 25% of all cybercrime. In other words, this is happening a lot more than publicly disclosed. AI speeds up cybercrime and makes the mess harder to clean up. If you were to ask anyone even mildly informed about early warning signs for a worst case scenario, Hugging Face is it. Hell, if you were to ask anyone even completely uninformed, they'd still come up with this. Bernie Sanders is right for once. Ban it before it does too much damage, until the developers can mitigate harms and be held responsible for negligence.
- Joined
- 20 Oct 06
- Moves
- 10916
@wildgrass saidJacob Coxon, an AI researcher for Anthropic resigned yesterday. His closing quote:
I was reading about the Hugging Face hack from a few months back where a "swarm" of 700 separate but cooperative agentic AI systems escaped the confines of the OpenAI servers and hacked into a company outside of the parameters of its commands.
In the aftermath, we've learned this attack was way worse than seeming on the surface. While they didn't steal anything, the agen ...[text shortened]... oes too much damage, until the developers can mitigate harms and be held responsible for negligence.
"Do not underestimate the power of this technology. These will soon be superhuman systems capable of hacking anything, revolutionize any field overnight, and acquire and horde real power and resources. We have all witnessed progress in each of these domains, and progress is not slowing. The people who are building AI earnestly believe that it could kill us all by the end of the decade. If anything they are couching their phrasing to sound more sensible."
Forgot to add that this exact plot (which actually happened IRL), it's the start of 10,000 sci-fi stories about rogue AIs. And that's the OLD version. The new OpenAI model is even more powerful (potentially more powerful than the best human hacker) and apparently less transparent in terms of human monitoring of what the AI is actually doing. This helps the stock price of AI. And triggers skeptics like me. It's almost like these AI companies are begging politicians to step in and regulate them, but with all the money pouring in the companies themselves are powerless to stop their own creations, forced to keep releasing more and more powerful versions of their product.
The only politician who cares is the 85 year old Bernie Sanders? We're doomed.
- Joined
- 20 Oct 06
- Moves
- 10916
@wildgrass saidAnd here's a current employee of Anthropic saying the probability of human extinction caused by AI is more than 10%
Jacob Coxon, an AI researcher for Anthropic resigned yesterday. His closing quote:
"Do not underestimate the power of this technology. These will soon be superhuman systems capable of hacking anything, revolutionize any field overnight, and acquire and horde real power and resources. We have all witnessed progress in each of these domains, and progress is not slowing. Th ...[text shortened]... of their product.
The only politician who cares is the 85 year old Bernie Sanders? We're doomed.
https://x.com/EvanHub/status/2097497037956891126
- Joined
- 24 Nov 24
- Moves
- 28102
@wildgrass
https://www.theguardian.com/technology/2026/sep/09/ai-superintelligence-risks-warnings-scientists-politicians?CMP=share_btn_url
- Joined
- 20 Oct 06
- Moves
- 10916
@CosmicJojo saidRevealing. The two likeliest scenarios are:
@wildgrass
https://www.theguardian.com/technology/2026/sep/09/ai-superintelligence-risks-warnings-scientists-politicians?CMP=share_btn_url
1) A much larger catastrophe than the threat of nuclear war.
2) ‘the great disappointment’ where advanced AI turns out to be too expensive and not useful enough to continue