The models aren't the product

Until this week, every answer in Too Many Requests went straight from a GPU pool to the Customers. A model answered, a customer paid, and that was the whole business.

Most people don’t talk to a model. They use an app built on top of one. So now the game has products.

A product sits on the wire between a pool and the Customers. Answers of its type pay more when they pass through it, and every other answer goes through untouched. There are four, one each for chat, images, code and agents, and each one has a catch.

The Assistant App makes chat pay half again. But users ask follow-ups, so some answers come back as new chat requests.
The Image Studio makes images pay 60% more. It upscales every picture, and you have to lend it GPU time for that.
The Cloud IDE makes code pay 80% more. It runs your tests, and the tests come back as agent requests, so you need a pool that can answer them.
The Agent Platform makes agents pay 60% more. It uses data as memory, so it needs a wire from a data source.

The GIF shows each product with its extra pay and its catch. A product that runs short on GPU time or data still passes every answer on, but only the answers it can cover earn the extra pay. Each product can also pass only so many answers a second, and past that it holds back the pool behind it. Where you put products and how many you build is a routing problem.

The Video Studio is the odd one. It cuts a video request into four image frames and sends them to your image pools. When four good images come back, it sends one video on. That costs more GPU time than a video model would, but you can serve video before you’ve trained one. The game describes it as “a video is only pictures, very fast”.

The Video Studio, four image frames to a videoFull size

All of this needed room in the research tree. It had fifty nodes in late September and has ninety-five now, so I changed its layout. It’s a grid now, with six lanes like Models, Data and Product, and eight tiers plus a finale. Each lane has one next step, so you can always see what comes after the thing you just bought. Each tier has one key node, and the next tier opens when you’ve met the tier’s goal and learned that node.

The research tree open at tiers 2 to 4, with the Power Grid, the Power Plant, the Assistant App and the Image Studio in their lanes — The tree at tier 3Full size

Most of the new nodes are upgrades, and twelve of them push the products further.

Power used to arrive all at once in tier 3. Now it comes in steps. A Generator in tier 1 runs two GPU pools, a Power Grid in tier 3 runs four, a Power Plant in tier 4 can run the bigger H-Class pools, and a Nuclear Plant comes in tier 7.

A Generator wired to two GPU pools — Tier 1. A Generator runs two GPU poolsFull size

A Power Grid wired to four GPU pools — Tier 3. A Power Grid runs fourFull size

A Power Plant wired to four H-Class pools — Tier 4. A Power Plant runs four H-Class poolsFull size

A Nuclear Plant wired to four Campus pools — Tier 7. A Nuclear Plant, here running four Campus poolsFull size

I added products this week, so every number here is a first guess. I don’t know yet if they make routing more interesting or only add more boxes to the floor. I’ll find out in the next playtests.