TL;DR: A semantic layer stores what your business terms mean, how data connects and who can see it, so an AI model answers from your definitions, not a guess. Start with one domain, define about ten terms, enforce access in the layer and keep it in version control.
A semantic layer for AI sits between your raw data and the models that use it. It says what your terms mean (revenue, active user, qualified lead), how tables connect and who can see what. So the model answers from your definitions, not a guess. If you are building an AI product, it is cheap to do early and costly to bolt on later.
Here is the idea in plain words, how an agent uses it, what goes inside and how to build one.
What a semantic layer is
Your database knows column names like `amt_net_v2` or `cust_status`. It does not know that finance reports revenue after refunds. It does not know a "customer" only counts after the first paid invoice. The semantic layer holds that knowledge.
MIT Sloan describes it as a system of technologies and techniques that keeps a consistent, unified view of data from different sources. It sits between the data and the people or machines using it. It explains what the data represents, how the pieces relate and which rules govern its use.
A few terms, one line each:
- Metric definition: the exact calculation behind a number, with its filters and time window.
- Taxonomy: an agreed list of categories, such as product families.
- Ontology: a description of the things in your business (customer, order, policy) and how they connect.
- Knowledge graph: your real data stored as those things and links, so software can follow them.
You do not need all four on day one. A short, accurate list of metric definitions and entity links is already a real semantic layer.
Why LLMs need one
A general model has never seen your business. It does not know if "monthly" means a calendar month or a rolling 30 days. It does not know if "active user" means someone who logged in or someone who did something.
The failure is quiet. The model still writes a query. It still returns a number. It sounds just as sure as when it is right. Nobody notices until two reports disagree in a board meeting.
Here is why that matters for a founder. If your in-app assistant tells a customer their churn rate, stock level or invoice total, that number must match the rest of your product. A semantic layer makes that true by design. The other way is to test prompt after prompt and hope.
How an AI agent uses the layer
An agent works with the layer in two steps.
- Grounding. Before it writes a query or acts, the agent reads the layer. Which metrics exist, what they mean, how they can be split and which rules apply.
- Execution. The agent asks for data through the layer, using the approved definitions. It does not write free-form queries against raw tables. Security rules travel with the request, so the agent only sees what the user it acts for may see.
The second step is where governance becomes real. Put row and column rules in the layer and every consumer inherits them: dashboards, notebooks, chatbots and agents. Keep them in each app instead, and every new AI feature is a new chance to leak data. We go deeper in our guide to data governance for AI.
Standards like the Model Context Protocol (MCP) help agents reach your systems. They do not supply meaning. IDC makes the same point: a semantic layer is the foundation of shared business meaning, and it goes beyond protocol-level interoperability. Connection alone does not give agents a common understanding.
Semantic layer or context layer?
You will see both terms. The difference is who uses it.
| Semantic layer | Context layer | |
| Main user | Analysts and BI tools | AI agents, at the moment they answer |
| Typical content | Metrics, dimensions, filters, joins | All of that, plus documents, policies, permissions and live data |
| How it changes | Updated when a definition changes | Updated all the time from live sources |
| Typical failure | Two dashboards disagree | An agent answers with confidence and is wrong |
Treat the semantic layer as the foundation. Add the context layer as agents take on more work. A human analyst knows which edge case applies. An agent does not, so the rules a person keeps in their head must be written down where the agent can read them.
For most Australian startups, the move is simple. Build the semantic layer first. Design it so agent context can be added later. Skip the big knowledge-graph program until one use case works. For more on this idea, see context engineering from AI-Led and the context stack for business from AI Orchestrators.
What to put in it
Keep the first version small and strict. Give every item an owner.
- Business names and descriptions. Plain names for metrics and entities, with a sentence on what each means and does not mean.
- Synonyms. Map how people really ask ("sales", "turnover", "takings") to one metric.
- Metric definitions with filters and joins built in. "Net revenue" should always treat refunds the same way.
- Entity links. Customer to account to subscription to invoice, with how many of each.
- Access rules. Who sees which rows and columns, tied to the metric, not to a raw table.
- Approval status. Mark which definitions are approved for AI and which are drafts.
- Lineage and freshness. Where the number comes from and when it last updated.
If your product handles personal information, access rules and logging are not optional. Our overview of AI software compliance covers what to plan for. Scopes for agent access are in our OAuth scope checklist. The AI Orchestrators data governance guide is a good companion.
Where it sits in the platform
The semantic layer is one layer in a bigger AI platform. It sits between your data stores and your models, agents and apps. AWS Prescriptive Guidance lays out a pattern with data, knowledge, intelligence and orchestration layers. Agents reach the knowledge layer through standard interfaces like MCP and A2A. It also mixes fixed reasoning rules with LLM output, so the reasoning can be audited.
You do not need the full stack. The idea scales down. Keep meaning in one place, expose it through a stable interface and let the model call it. Our AI platform architecture guide shows where the layer fits. Our integration patterns for SaaS AI show how to connect it to a product you already run.
How to build one
This is the order we would follow for a team with a live or planned AI feature.
1. Pick one domain and one use case
Choose the place where a wrong number costs you most. A customer report, a finance assistant, a support agent quoting order status. Do not model the whole company. One domain keeps the first release to weeks, not quarters.
2. Write down the questions first
Collect the 20 to 30 questions people really ask of the data. Underline each business term. Those are your first definitions. If two people define a term differently, you have found the first thing the layer must settle.
3. Define each term once, so it can be tested
For each metric, write the calculation, the filters, the time window, the grain (per day, per customer, per order) and the owner. Add two or three example questions with the right answer. These become automated tests.
4. Bind definitions to real tables and enforce access
Map each definition to the warehouse tables that feed it. Attach row and column rules here. An agent should never get a way around them. A solid data warehouse for AI makes this step much easier.
5. Expose it through a stable interface
Offer the layer through an API or an MCP server. Then agents, dashboards and app code all read the same definitions. Prefer open standards over anything that locks you to one tool.
6. Treat it as code
Keep definitions in version control. Review changes. Run your example questions in CI. Deploy it like app code. When a definition changes, you want to know which answers changed with it. The same care applies to the models, as we cover in AI model versioning in SaaS.
7. Measure and extend
Track how often the agent picks the right metric, how often it refuses and how often users correct it. Add the next domain only when the first is stable.
Common mistakes
- Treating a BI semantic layer as agent-ready. Dashboard definitions assume a human will judge edge cases. Agents need those exceptions written down.
- Letting each feature carry its own definitions. Within a quarter the chatbot, the dashboard and the export each count "active customer" differently.
- Putting security in the prompt. An instruction to a model is not access control. Enforce rules where the data is queried.
- Modelling everything up front. A huge ontology with no use case goes stale before anyone depends on it.
- No owner. Every definition needs a person who can say what it means and approve a change.
Build it as part of your AI platform
Devwiz builds AI agents, AI-first platforms, custom apps and the data and automation underneath them. We have shipped 200+ apps, including work for NSW Government, Briometrix, Vivid and Huskee. Our process runs from discovery to launch and ongoing support.
Want a semantic layer designed together with the agents, apps and data pipelines that rely on it? See our AI platform development service.
Already running a SaaS or mobile product and want AI on top? Our AI automation page explains how we build AI into existing software. Our AI agents overview covers what agents need from the platform beneath them.
Frequently asked questions
What is a semantic layer in the context of AI?
It is a governed layer that tells AI models what your data means: metric definitions, how things relate and the rules for using them. The model reads it before answering and queries through it, so results follow your definitions and permissions, not its guesses.
Do I need a semantic layer for an LLM feature?
If the feature answers questions about your own data, yes, in some form. Even a short list of metric definitions with clear owners improves consistency. A feature that only drafts text from user input needs far less.
Is a data warehouse the same as a semantic layer?
No. A warehouse stores and organises data. A semantic layer sits on top and adds meaning: which calculation is revenue, how tables join and who may see what. A solid warehouse makes the layer easier to build.
How long does a first version take?
It depends on how many definitions are disputed and how clean the data is. One domain and one use case keeps the first release to weeks. Discovery on your real data gives a firm estimate.
Is a semantic layer the same as RAG?
No. Retrieval-augmented generation (RAG) fetches documents for a model to read. A semantic layer supplies approved definitions and structured access to data. Many products use both: RAG for documents, the semantic layer for numbers and records.
About James Killick
10+ years building digital products · 200+ apps shipped since 2015
James is a co-founder of Devwiz and an AI product specialist. Since 2015 he has helped ship 200+ apps for founders, businesses and government, including work for NSW Government, Briometrix and Huskee. He builds AI-first platforms and writes about turning a proven program into software. He also hosts the Up in the AI podcast.
More articles by James · James's personal site · LinkedIn · AI Orchestrators
Tags: Semantic Layer, AI Platforms, Data Governance, LLM


