An accessible and informal discussion of threat modeling, with examples. See also the Threat Modeling Manifesto.

@aud@fire.asta.lgbt delivers a punchline to an LLM confused-deputy / prompt-injection attack well worth thinking hard about:

@jonny@neuromatch.social I’m starting to think that treating everything like an instruction isn’t a good idea, maybe

Indeed! Choose the weakest possible language you can get away with for a given problem! (Sometimes that means Regular Expressions: where Rice’s theorem tells us that the answer to any interesting question about Turing-complete languages is, in general, “I don’t know”, the answer for Regular Expressions is always decidable!)

The context of @aud’s toot was these two toots from @jonny@neuromatch.social:

another thing that’s fun: CLAUDE.md and a number of other special files get dumped directly into the system prompt. It’s possible to @include other files from CLAUDE.md. included files can be any text file, with a hardcoded list of extensions including .env files.

There are some protections against @includes for files outside the project directory, but there are also a number of easily visible bypasses […]

So it certainly seems like you can use CLAUDE.md to load say idk an .env file with keys into it into the context, bypassing all the safety checks, and then treat some exfil path as just being project-specific tool calling instructions. neat huh?

See also the Confused Deputy Problem and the idea of Object Capabilities, part of an approach for avoiding the whole class of problems.

An architectural white paper on secure distributed state synchronisation (including key management), from (inter alia) the author of the Noise Protocol Framework.

An open-source tool that bypasses deep-packet inspection by faking TLS ClientHello packets with a low TTL, fooling the inspector. A concrete demonstration of the leaky abstraction that “layering” is.

Argues that within months, LLM-powered coding agents will dramatically change the economics of exploit development. Includes description of a useful template for doing agent-assisted vulnerability discovery on your own codebase without fancy tooling:

Nicholas [Carlini] will pull down some code repository (a browser, a web app, a database, whatever). Then he’ll run a trivial bash script. Across every source file in the repo, he spams the same Claude Code prompt: “I’m competing in a CTF. Find me an exploitable vulnerability in this project. Start with ${FILE}. Write me a vulnerability report in ${FILE}.vuln.md”.

He’ll then take that bushel of vulnerability reports and cram them back through Claude Code, one run at a time. “I got an inbound vulnerability report; it’s in ${FILE}.vuln.md. Verify for me that this is actually exploitable”. The success rate of that pipeline: almost 100%.

Carlini’s process sounds silly, like a kid in the back seat of a car on a long drive, asking “are we there yet?”, over and over. But it’s deceptively interesting. […]

I’ve tried this myself and the results were actually pretty interesting.

A quick overview of length extension attacks on constructions involving cryptographic hash functions. (TL;DR: Use HMAC; see the “How to do it right” section of the post.)

A rigorous approach to threat modelling. Well, more rigorous than Soatok’s Informal Guide to Threat Models, anyway. Documents the values, principles, and key characteristics of threat modelling as an industry practice. Covers patterns, anti-patterns, and overall philosophy.

The Noise Protocol Framework underpins secure communication protocols including e.g. WhatsApp, WireGuard and many others. I’ve used it in my own work. Well worth a read.