Day 1 - How to Design AI Systems

Session Q&A - Are there any recommendations on determining/estimating costs?

We invite community to participate in this discussion. Thank you!

2 comments

Estimating costs in agentic systems is less about a single "price tag" and more about understanding the economy of your graph. Because agents operate in loops, the best way to get a real-world estimate is to run a representative test set and calculate the averages. You want to track exactly how many times each specific agent—your Researcher, your Programmer—is called on average to complete a task.

By measuring the average token consumption for each of those roles and multiplying them by their respective model prices, you can get a predictable baseline. It’s all about balancing that "frequency of calls" against the "token price" of the model assigned to that node. For a quick mental model, you can use this:

Total Cost = Σ (Avg. Calls per Agent × Avg. Tokens per Call × Model Token Price)

Forget a single price tag, think of it like managing a project team's budget. You're not paying one person a flat fee; you're tracking how many hours each specialist works and what their hourly rate is

In this agentic system, the "specialists" are your Researcher, Judge, and Content Builder. The "hours" are the LLM calls and the tokens processed. To build a realistic budget, you need to run a few projects (your test cases) and track the team's workflow.

So you wanna know how much this whole agent thing is gonna cost you? Let me give you the real tea.

Okay, so imagine you're running a little content agency. You've got:

  • Your Researcher (the one who's always on Google)

  • Your Judge (the picky editor)

  • Your Content Builder (the writer)

Here's the thing you don't just pay them a flat fee and call it a day. You're paying per task, and sometimes they have to redo stuff.

Here's what actually happens:

When you ask for a course on "Quantum Computing for Beginners":

  • Round 1: Researcher goes off, finds stuff. Judge looks at it: "Ehhh, this is too technical, fail." 🔄

  • Round 2: Researcher tries again, simpler this time. Judge: "Now we're talking! Pass!" ✅

  • Final: Content Builder writes the course. 📝

So how do you budget for this?

You gotta run some test runs first! Like, grab 10 different topics and see what happens:

  • How many times does your Researcher actually have to search on average? (Maybe 2.3 times)

  • How many tokens does each role chew through? (Researcher reads a LOT, Judge just says "pass/fail" with some notes)

  • Which model is each role using? (Your Judge can probably use a smaller, cheaper model - they're not writing essays!)

The simple formula I tell everyone:

Total Cost = 
  (Researcher calls × Researcher tokens × Researcher model price) + 
  (Judge calls × Judge tokens × Judge model price) + 
  (Content Builder calls × Content Builder tokens × Content Builder model price)

It's not about guessing one number. It's about understanding your team's workflow. Run the tests, track the patterns, then you'll know exactly what to expect.