Discussion about this post

User's avatar
Joel Holt's avatar

Rick, the “Morris Worm of Intelligence” analogy is a sharp and unsettling way to frame what decentralization really breaks. I’m fully aligned with the techno-optimist vision and agree that regulation alone can’t keep pace with the speed of code. Where I’m still wrestling is less with whether something like a SIRT is needed, and more with where the real leverage lives once autonomy and scale arrive together.

Even a hybrid “Navy SEALs” model risks being reactive by default. If decentralized agents propagate exponentially, a human-led response (even with Hunter AIs) may always be fighting a linear battle. Near-term defense may live as much in architecture as in teams: aggressively gating high-level inference upstream, enforcing identity or proof-of-personhood for certain capabilities, and building systemic friction that buys time. I don’t see that as an argument against SIRT—if anything, it defines the hard operational work such a team would need to own.

Kelley Bulgin's avatar

This proposal sounds rational to me. The caveat is that “hunter AI’s” are the kind of thing we also find in science fiction. The issue being as the maxim goes: “To a hammer, every problem is a nail.” And what happens if we have Hunter AI’s developed and tuned to such a high degree with the power to eliminate code, shut power off, fry circuits etc. and it decides to shut down critical infrastructure for a false alarm? I can imagine planes falling from the sky already. That kind of baked in instinct to act with prejudice- It’s the trope where in an effort to create a monster killer, we accidently create a monster. Maybe this is called the bigger gun is needed trope. In any case, I do appreciate the need and I’m glad smart people with agency are thinking about it and working on it.

4 more comments...

No posts

Ready for more?