Day 1 - How to Design AI Systems

Session Q&A - Are agents built from fine-tuning LLMs or from a different paradigm altogether

We invite community to participate in this discussion. Thank you!

2 comments

Agents are a design paradigm, not a specific model architecture.

Think of it this way: Fine-tuning is like giving someone a specialized education or a better vocabulary. You’re changing the "brain" itself so it knows specific facts or follows a certain tone.

Agency, on the other hand, is about the nervous system and the tools you give that brain. You don't build an agent by retraining a model; you build it by wrapping a powerful LLM in a framework like LangGraph. You give it "hands" (APIs and tools), "memory" (state and databases), and a "reasoning loop" (the ability to plan and self-correct).

In short: Fine-tuning makes the model smarter at a specific subject, but Agentic Orchestration is what gives that model the autonomy to actually do something with that knowledge. Most modern agents use off-the-shelf, powerful models and focus all the innovation on the system architecture around them.

The core distinction is between knowledge and autonomy.

Fine-Tuning: This is an offline optimization. You are permanently altering the model's weights to embed specific knowledge, domain expertise, or a particular tone. It changes the "brain" to make it an expert in a subject (like law or medicine).

Agency (Orchestration): This is an online runtime architecture. You are not changing the model's brain; you are building a system around it. You give it tools (APIs), memory (state), and a reasoning loop (planning/reflection).

When you fine-tune a model, it's like sending it back to school. You're cramming its brain with specialized knowledge medical jargon, legal precedents, your brand's specific voice. You're making it smarter about one particular thing.

But an agent? That's not about changing the brain anymore. That's about building the whole nervous system.

You take a smart model off the shelf, no extra school requiredand you give it hands (that's APIs and tools). You give it memory (so it remembers what happened five steps ago). And most importantly, you give it a loop the ability to pause, think, realize it messed up, and try a different approach.