AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics. https://www.csoonline.com/article/4234854/ai-agents-arent-going-rogue-frontier-labs-are-skipping-security-basics.html
Mastodon Digest
Posts
Three modern annoyances that I'm actually quite happy to suffer through because I know how severe the consequences they're mitigating are:
1) Flight delays to fix mechanical issues
2) Power outages because wildfire safety mechanisms tripped
3) Getting vaccines
“Cyber hygiene would have prevented the access: take the controller off the public internet, put remote access behind a VPN and an IP allowlist, bring third-party kit onto the network in a controlled, owned and documented way rather than leaving it on a contractor’s own connection, change known and default credentials, and patch to a current release.”
Yes, it’s a sane suggestion in principle. Yet we live in a world where every toaster is connected to the public internet, sends data to its vendor (probably not only them) and never gets any patches installed – even if it is lucky enough to be still supported a year after being manufactured. Most people are buying this crap for their homes without a second thought, so why wouldn’t they do the same for their companies? I don’t think the problem is going to go away unless governments start to regulate these markets.
I’m still missing one piece of the puzzle here however. It was a coordinated attack against hundreds of devices at the same time. How did the attackers find them? I doubt that these controllers are on Shodan. Did they scan the internet for months to prepare the attack? That would rule out vandalism, that’s the kind of preparation a nation-state actor would do. Or did they hack the vendors who tend to be a sweet target with knowledge of where all the vulnerable devices are?
Fascinating case, that OpenAI employee who tried to wiretap executive meetings: https://cdn.arstechnica.net/wp-content/uploads/2026/10/US-v-Alfozan-Complaint-10-8-26.pdf
So he used a chatbot to advise him on how to build a spy device from an Arduino-like board. He solved the problem of getting power to it by plugging it into a USB port of a TV, and he solved the issue of getting data out with a 2.4 GHz antenna. My reading seems to indicate that his hardware was capable of WiFi communication, so presumably this allowed it to be accessed remotely. Which would also explain why he showed up immediately when his device was removed – he wanted to check why it stopped transmitting.
I wonder why he thought that he would succeed at recording anything. Clearly he didn’t use the camera – in the photos it isn’t attached to the device, and it would be of no use behind a TV anyway. As to the microphone, I found the following statement:
The integrated MEMS microphone on the XIAO ESP32-S3 Sense is not designed for studio-quality recording, but it delivers usable voice data at close range (under 1 meter) when paired with proper signal processing.
It was definitely a lot more than one meter in that meeting room, and there was a TV in the way distorting the sound. Besides, I’ve seen how difficult it is to get useful recordings in a meeting room even with expensive hardware far larger than his tiny board. Didn’t he test its recording quality, or did he hope that he would be able to “clean up” the recording afterwards? Maybe that a chatbot would be able to decipher the spoken words?
Did I already say that LLMs make people wildly overestimate their skills?
Roc et Tête de Garnesier
Boosts
Ca m'évoque exactement la même chose qu'un dev qui fait un outil de manipulation des PDF, besoin que tout le monde a, dans son coin avec Claude. D'un côté t'as un type tout seul qui a pu, au prix de ressources importantes dont il ne paie pas les externalités négatives, créer un outil de manipulation des PDF pour lui tout seul.
De l'autre coté t'as framasoft qui déploie une solution existante, se joint aux 40 contributeurs pour l'améliorer et promeut une approche communautaire de la solution