What is the difference between an LLM, RAG and an AI agent?

An LLM is the model that understands, writes and reasons, but it only knows what it learned in training. RAG adds a lookup step, so the model searches your documents before answering and grounds the answer in your own data. An AI agent plans steps on its own and uses tools to act, not just answer.

MCP, the Model Context Protocol, is one standard plug that connects AI to tools and data. It lets a model reach apps, APIs, files and databases through a common interface, instead of a custom connection for each one. Think of it as the nervous system of an AI stack.

Skills are saved know-how for tasks you repeat, such as the steps, checks and format a task needs. A skill loads only when a task needs it, so the AI picks up the right instructions at the right moment instead of carrying all of them all the time.

Why does RAG matter for product teams?

A model on its own can't see your customers, policies or research. RAG lets it look up your documents before answering, so the answer is grounded in your data. Its quality depends on the library: stale or messy documents give confident answers from old information.

What should a product manager know before building an AI agent?

Know what each part of the stack does, so you can tell a reasoning problem from a data, connection or limits problem. Above all, decide where the agent stops and asks a person. An agent with clear limits does real work for you; one without them is a risk.

Do product managers need to know how to build an AI stack?

No. A product manager doesn't need to build each part, but does need to know which part does what. That is what lets you write a clear spec, ask the right question when something fails, and explain the trade-offs to leadership.

Your AI Stack Explained Like a Human Body: LLM, RAG, MCP, Skills and Agents

1

The full infographic, as a PDF

The exact one from the post, sized to print or keep within reach.

2

A copy in your inbox

The download link lands in your email too, so it's there when you need it. No follow-up campaign.

3

The Product System, weekly

My note on leading product in the AI era. One idea a week that compounds. Unsubscribe anytime.

Your AI stack, explained like a human body.

Before you build an agent, know what each part does.

Most conversations about AI at work mix five different things into one word. Someone says "the AI got it wrong", and nobody can tell whether the model was weak, the data was missing, the connection to a tool broke, or the agent took a step it shouldn't have. Each of those has a different fix.

For a product manager, that matters. You are the one writing the spec, setting the scope and explaining to leadership why it works or doesn't. You don't need to build every part. You do need to know which part does what.

The human body is a useful way to hold it in your head. Five parts, five jobs.

1. LLM: the brain

The large language model is the brain. It understands, writes and reasons. Ask it a question and it can explain, summarise, draft or argue a case.

But it only knows what it learned in training. It doesn't know your customers, your roadmap or what your team decided last Tuesday. Left on its own, it will answer from general knowledge, and it can sound confident when it is guessing.

The product question here: is the problem you are solving a reasoning problem, or a knowledge problem? A better brain helps with the first. It does nothing for the second.

2. RAG: brain plus library

Retrieval-augmented generation, or RAG, gives the brain a library. Before it answers, it looks up your documents and pulls in the relevant parts. The answer is then grounded in your own data, not just in what the model learned in training.

RAG is how you get an assistant that can answer questions about your product docs, your policies or your research notes.

The product question here: which documents should it trust, and who keeps them up to date? RAG is only as good as the library behind it. A stale or messy library gives you confident answers from old information.

3. MCP: the nervous system

MCP, the Model Context Protocol, is the nervous system. It is one standard plug for tools and data. Instead of building a custom connection for every app, MCP gives the AI a common way to reach apps, APIs, files and databases.

The brain can think. The library lets it know. The nervous system lets it reach things outside itself.

The product question here: what should the AI be allowed to reach, and what should it only read? Every connection is a decision about access, not just a technical setup.

4. Skills: muscle memory

Skills are muscle memory. They are saved know-how for tasks you repeat: the steps, the checks, the format, the things you always do the same way.

The useful part is that a skill loads only when a task needs it. The AI doesn't carry every instruction into every conversation. It picks up the right know-how at the right moment, the way you don't think about how to type once you have learnt.

The product question here: which of your team's repeat tasks have a right way of being done? Those are your first skills. Writing them down is also writing down your team's judgement.

5. AI agent: brain plus hands

An AI agent is the brain with hands. It plans the steps on its own and uses tools to act, not just answer.

A chat assistant tells you what to do. An agent does it: looks something up, calls a tool, checks the result, takes the next step.

The product question here is the biggest one: where does the agent stop and ask a person? An agent with no stopping point is a risk. An agent with clear limits does real work for you. If you are writing a spec for one, The Missing Line in Your Agent Spec (/vault/the-missing-line-in-your-agent-spec) covers the line most specs leave out.

How the parts work together

None of these parts is the whole system. They stack.

The brain reasons. The library grounds it in your data. The nervous system connects it to your tools. Muscle memory makes repeat work reliable. The hands let it act.

When something goes wrong, the body view helps you find where. A wrong fact is usually a library problem. A failed action is often a nervous system problem. Inconsistent results on a repeat task point to missing muscle memory. And an agent that did something nobody wanted is a hands problem: the limits were never set.

That is the real value for a product manager. Not knowing the engineering, but knowing which question to ask about which part. For the wider vocabulary, AI Terms for Product Managers (/vault/ai-terms-for-product-managers) covers the 20 terms that set direction on a roadmap.

Every product manager has an AI profile. Find out yours at martongaspar.com/scorecard.

Questions people ask

What is the difference between an LLM, RAG and an AI agent?: An LLM is the model that understands, writes and reasons, but it only knows what it learned in training. RAG adds a lookup step, so the model searches your documents before answering and grounds the answer in your own data. An AI agent plans steps on its own and uses tools to act, not just answer.
What is MCP in AI?: MCP, the Model Context Protocol, is one standard plug that connects AI to tools and data. It lets a model reach apps, APIs, files and databases through a common interface, instead of a custom connection for each one. Think of it as the nervous system of an AI stack.
What are AI skills?: Skills are saved know-how for tasks you repeat, such as the steps, checks and format a task needs. A skill loads only when a task needs it, so the AI picks up the right instructions at the right moment instead of carrying all of them all the time.
Why does RAG matter for product teams?: A model on its own can't see your customers, policies or research. RAG lets it look up your documents before answering, so the answer is grounded in your data. Its quality depends on the library: stale or messy documents give confident answers from old information.
What should a product manager know before building an AI agent?: Know what each part of the stack does, so you can tell a reasoning problem from a data, connection or limits problem. Above all, decide where the agent stops and asks a person. An agent with clear limits does real work for you; one without them is a risk.
Do product managers need to know how to build an AI stack?: No. A product manager doesn't need to build each part, but does need to know which part does what. That is what lets you write a clear spec, ask the right question when something fails, and explain the trade-offs to leadership.