Jadi gini, AI companies kayak Anthropic dan OpenAI udah ribet banget bikin guardrails buat prevent penyalahgunaan AI mereka. Tujuannya jelas: biar gak ada hacker nakal yang pake AI buat serang sistem orang. Tapi realitanya? Policy ini justru jadi red flag besar buat legitimate network defenders dan offensive cybersecurity researchers.
Government Intervention: Mythos & Fable Kena Ban
Cerita mulai serius pas pemerintah AS impose export control restrictions ke model AI Anthropic, Mythos dan Fable, di Juni lalu. Intervention ini dipicu laporan yang bilang model-model ini bisa di-bypass buat build dan execute malicious cyberattacks. Entah ini motivasi utama atau bukan, yang jelas Anthropic emang dari awal udah market Mythos sebagai "doomsday cyber machine" yang cuma boleh dipakai sama vetted users dengan guardrails ketat. Export control udah dicabut sekarang, tapi Fable 5 balik ke general access baru Juli lalu, dan Mythos 5 masih restricted cuma buat U.S. organizations yang udah diverifikasi.
Vetted Programs: "Treating Us Like Children"
Kedua perusahaan ini emang punya program cybersecurity research yang bisa di-apply: OpenAI's Trusted Access for Cyber dan Anthropic's Cyber Verification Program. TAPI, program-program ini malah kena kritik pedas dari researcher sendiri.
Mark Dowd, security researcher yang udah puluhan tahun найден zero-days buat Western governments, bilang di podcast cybersecurity:
"It's not really comfortable to me that these random large companies are making arbitrary decisions about what is safe in security and what's not."
Meanwhile, Paolo Stagno dari Crowdfense (perusahaan yang specialize in acquiring dan selling unknown vulnerabilities ke government agencies) ngomong lebih blunt:
AI companies "essentially treat customers like children who need babysitting"
Chris Anley, chief scientist di NCC Group, nge-breakdown problemnya dengan非常好 analogy:
"'Fix this code' as a prompt is both an essential mechanism for defense but also a roadmap for finding critical vulnerabilities in the codebase. So at the same time, the same tool is both an offensive tool and a defensive tool, and the two can't really be unpicked."
"It's like a hammer," katanya. "You can't build a house without a hammer. It's definitely a tool but it's also irreducibly a weapon as well."
The Consequence: Researcher Dipaksa ke Open Source
Nah ini yang bikin concerned. Ketika researcher udah ke-blok sama guardrails, mereka biasanya fallback ke open source AI models yang literally gak punya restrictions sama sekali. Jadi bukannya mitigate risk, guardrails ini malah ngarahin researcher ke tools yang potencialmente lebih risky.
Chris Thompson, CEO RemoteThreat dan founder Offensive AI Con, bilang pengalaman dia sama frontier AI models:
"I think the practical impact is you spend a lot of time negotiating with the model instead of working on the core security program. Instead of analyzing a vulnerability and reasoning through the exploitability, you're trying to find why you're getting inconsistent results or why are models over-sanitizing the output."
Ini yang diautinconsistent: guardrails frontier models kerja beda-beda setiap hari, bahkan di dalam vetted programs yang udah "loosened".
Hasilnya? Researcher sekarang rely on Chinese open source models kayak GLM yang bisa didownload bebas dan run locally tanpa vetting atau usage restrictions. Thompson warning:
"You have these responsible researchers that are being pushed away from U.S.-governed systems to foreign-owned systems. I think it's more harmful than good to have these guardrails in place."
Geek Opinion: Stop Babying Us, Trust the Pros
Oke jadi ini serius. Guardrails yang dimaksudin buat ngelindungi publik malah bikin defender keok dan ngarahin researcher ke sistem asing. Gak masuk akal.
Thompson呼吁 AI labs buat:
- Open up programs dan kasih responsible access
- Trust legitimate researchers
- Hold abusers accountable daripada block semua orang preemptively
"There's this big storm coming. There's this big wave of attacks that are going to happen at speed and scale like never before. But the same security consulting firms and legit researchers that are trying to make a difference are being stifled right now."
Intinya: AI companies need to stop treating cybersecurity researchers como threats. Researcher kayak Dowd, Anley, dan Thompson udah proven track record mereka. Mereka butuh tools yang work, bukan babysitting.
Sointinya, kalau kita terus restrict access legitimate researchers, yang terjadi adalah:
- Defender kehilangan AI advantage mereka
- Researcher professional pushed ke foreign systems
- Attackers (yang emang gak bakal pake legal channels) tetep dapet tools mereka
Jadi siapa yang diuntungkan? Probably bukan kita.
It's time to rethink this approach.
