A lesson on ToC and LLMs spitting out code faster than you can read it
To all the “the rate of code generated by agents far exceeds the rate at which humans can review it, so if you want to understand the generated code, you become a bottleneck” takes out there. As if being called a bottleneck were an insult 🤷
Every system has a constraint (colloquially known as a bottleneck). Every single one. Without one, it would have infinite throughput. The fact that no system does is proof that every system has one.
For those not familiar (enough) with the Theory of Constraints: it never said bottlenecks are bad per se. It says they are bad when they’re unmanaged, including not placing them in the right place in the system. There are plenty of examples where placing a bottleneck at the right place, counterintuitively, increases the throughput of the whole system. Imagine that! 😱
Randomly speeding up one stage until inventory (code, in this case) piles up in front of the next one doesn’t make the next stage the problem. It means the previous stage is running too fast, and you need to slow it down if you want to manage the flow through the whole system.
And if speeding up a stage didn’t increase the throughput of the whole system, it was never the bottleneck. You’re just piling up more inventory, which slows everything down even further.
Here’s a Discrete-Event Simulation (DES) I built a couple of months ago to demonstrate the point and build intuition for thinking about systems through a ToC lens.