
Retrieval-augmented generation (RAG) is a way of building AI applications so that, before a language model answers a question, the system first searches your own documents and gives the model the most relevant passages to answer from. It lets an AI assistant use private, current information and cite its sources without retraining the model.
A production RAG system for a US business typically costs $15,000 to $40,000 as a single-source pilot, $40,000 to $100,000 for a production system with several sources, permissions and evaluation, and $100,000 to $250,000 or more for an enterprise, multi-tenant or regulated build. Running costs range from a few hundred dollars to several thousand dollars a month. And RAG is not always the right answer: if your whole knowledge base fits in about 200,000 tokens (roughly 500 pages) and gets few questions, putting it straight into the model's prompt can be cheaper and simpler.
Almost every company with a pile of documents has had the same idea: "let people ask questions and get answers from our own files." Vendors call it RAG, and some call it enterprise search or a knowledge assistant. It is a good idea, and it is also one of the easiest AI projects to build badly. The demo works on ten documents. Then it meets ten thousand, with outdated versions, scanned PDFs and files that only some employees should see.
This guide is for the person responsible for getting it to work: a CTO, a head of operations, a product owner. We cover when RAG is the right tool and when it is not, the architecture that holds up in production, what it costs with an original per-query cost model, the security issue most demos ignore, and the failure modes to plan for.
We build AI systems for a living, so we have a point of view. Where the cheaper answer is not to build RAG at all, we say so.
Ingest. Your documents are collected, cleaned and split into passages called chunks.
Index. Each chunk is converted into a numeric representation (an embedding) and stored in a search index, often a vector database.
Retrieve. When a user asks a question, the system searches the index and finds the most relevant chunks.
Generate. The question and the retrieved chunks go to a language model, which writes the answer.
Cite and check. The system shows which passages the answer came from, so a person can verify it.
OWASP's LLM Top 10 project lead has said that some form of RAG is now the default architecture for enterprise language model applications, and the project added a dedicated entry on vector and embedding weaknesses to its 2025 Top 10.
Approach | What it does | Best for |
|---|---|---|
RAG | Searches your documents at question time and gives the model the relevant passages | Large, changing, private knowledge bases that need source citations |
Long-context prompting | Puts the whole document set into the prompt | Small, stable document sets, or deep reasoning across a few long documents |
Fine-tuning | Adjusts the model itself with your examples | Style, format and behavior, not facts that change |
Traditional search | Returns a list of matching documents | When users want to read the documents themselves |
SQL or API queries | Retrieves exact records from structured systems | Numbers, balances, inventory and anything that must be exact |
Anthropic's own guidance in its Contextual Retrieval write-up is that if your knowledge base is smaller than about 200,000 tokens, roughly 500 pages, you can simply include the whole thing in the prompt, ideally with prompt caching. Frontier models now accept far more than that: a June 2026 research paper notes that OpenAI's GPT-5.5 supports a context window of about 1.05 million tokens, and frames the choice between long context and RAG as a trade-off between accuracy and the recurring token cost of reading more. The cost model below shows how big that cost gap is.
RAG finds passages of text. It is a poor tool for "what was our revenue in Q3" or "how many units are in stock," which belong to a database query. Use an AI layer that writes and runs a query against structured systems, and keep RAG for the unstructured material around it.
If the use case is standard, such as customer support or an internal helpdesk, a packaged product may do it for a published per-resolution price. Our guides to custom software vs SaaS and AI agent development for US businesses cover the build-or-buy math.
RAG reduces hallucination and does not eliminate it. If a wrong answer causes harm, such as a medical dosage or a legal clause, the system needs human review or a deterministic path, with RAG as a helper rather than the decision-maker.
Situation | Usually the better choice |
|---|---|
Under about 200,000 tokens of stable content, low question volume | Long-context prompting with caching |
Hundreds of thousands to millions of tokens, content changes often | RAG |
Questions about numbers in structured systems | SQL or API queries, with AI as the interface |
Standard support or helpdesk use case | A packaged product first |
Deep reasoning across a few long documents | Long context, possibly with retrieval to narrow the set |
Strict latency target of about two seconds | RAG |
Most RAG projects succeed or fail here. Messy PDFs, scans, tables and slide decks need parsing that preserves structure, and chunks must be sized so that each carries enough meaning to be found and understood. One 2026 cost guide estimates that data preparation and ingestion are 40% to 50% of the total engineering work, which matches what teams discover the hard way.
Embeddings find passages with similar meaning and can miss an exact term such as an error code or a policy number. Keyword search (BM25) catches those. Production systems combine both and merge the results.
A chunk such as "revenue grew 3% over the previous quarter" is useless without knowing which company and which quarter. Anthropic's Contextual Retrieval method prepends a short explanatory context to each chunk before indexing. In its tests, the top-20 retrieval failure rate fell from 5.7% to 3.7% with contextual embeddings, to 2.9% when combined with keyword search (a 49% reduction), and to 1.9% when a reranking step was added (a 67% reduction). Those are one company's results on its own test sets, so measure on your own data, but the order of the gains is a useful guide to where effort pays.
Instruct the model to answer only from the retrieved passages, to say when it cannot find an answer, and to cite its sources. Showing the source passage is the cheapest trust mechanism you have.
The security section below covers this in detail, but design it in from day one. Retrieval must respect who is allowed to see what.
Measure three things: whether the retrieved passages are relevant to the question, whether the answer is supported by those passages, and whether the answer addresses the question. Build a test set of real questions with known answers before launch, and rerun it after every change. A system with no evaluation set is a system nobody can improve.
Log questions, retrieved sources and user feedback. Watch for stale documents, questions the system cannot answer and drift after model changes. Index freshness is an operations job, not a one-time setup.
These are our planning figures, built from the published ranges in the next section and sized for the US market. They are not a quote.
Tier | Build cost | Typical timeline | Monthly running cost | What it usually includes |
|---|---|---|---|---|
Single-source pilot | $15,000 to $40,000 | 4 to 8 weeks | $300 to $800 | One document source, basic chunking, vector search, a chat interface, citations. |
Production system | $40,000 to $100,000 | 10 to 16 weeks | $1,000 to $3,000 | Several sources, hybrid retrieval and reranking, role-based access, audit logging, an evaluation set, integrations. |
Enterprise, multi-tenant or regulated | $100,000 to $250,000+ | 16 to 28 weeks | $2,000 to $13,000 | Large corpora, strict permissions, compliance work, agent workflows, on-premises or private cloud options. |
Source (2025 to 2026 guides) | What it says |
|---|---|
ScalaCode | $10,000 for a basic prototype to $150,000 or more for an enterprise agentic system. Vector database from $0 (pgvector) to $2,000+ a month. |
RaftLabs | A production RAG system costs $30,000 to $60,000 to build, with $300 to $2,500 a month in infrastructure. Simple pipelines take 4 to 6 weeks and enterprise platforms 14 to 22 weeks. |
Ortem | Simple RAG $15,000 to $25,000, production $40,000 to $80,000, enterprise on-premises $80,000 to $150,000+, with timelines from 4 weeks to 28 weeks. |
Dyyota | Enterprise RAG $40,000 to $150,000+ with $2,000 to $8,000 a month in ongoing costs. |
Huzaifa Tahir | $5,000 prototypes to $250,000+ enterprise platforms, with monthly operations from $200 to $13,000. |
Three things explain the spread. The guides define "RAG" differently, so a prototype on one document source shares a price list with a multi-tenant platform. They differ on whether evaluation, permissions and data cleaning are included. And nearly all are written by firms that sell this work. Use them to find the range, then test any quote against hours and roles, as we show in our custom software development cost guide.
Data preparation. Cleaning, parsing and structuring documents is the largest single piece of engineering effort.
Evaluation. Building the test set and the measurement pipeline is the line item vendors forget to quote.
Permissions and audit. Mirroring your access rules into search takes real design work.
The vector database, less than people think. pgvector on an existing Postgres database costs little and one guide reports it handles up to about 5 million vectors comfortably. Managed services such as Pinecone start at a $50 monthly minimum on the Standard plan and can reach thousands at large scale. One vendor reports that real vector database bills run 2.5 to 4 times the pricing calculator estimate once re-indexing and operations are included, so treat that as a vendor claim and budget a margin.
Model usage. It scales with questions asked, which the next section models.
Here is a simple model of what one question costs. Every input is an assumption you should replace with your provider's current prices: $3 per million input tokens and $15 per million output tokens, typical of mid-tier models in 2025 and 2026. A RAG query sends about 3,500 input tokens (five retrieved chunks plus the question and instructions) and gets back 400 tokens. A long-context query sends the whole document set.
Approach | Cost per query | 5,000 queries a month | 50,000 queries a month |
|---|---|---|---|
RAG (3,500 tokens in) | $0.0165 | $83 | $825 |
Long context, 200,000 tokens | $0.606 | $3,030 | $30,300 |
Long context, 200,000 tokens, fully cached at 10% of input price | $0.066 | $330 | $3,300 |
Long context, 1 million tokens | $3.006 | $15,030 | $150,300 |
Uncached, a 200,000-token prompt costs about 37 times more per query than RAG, and a 1-million-token prompt about 180 times more. Caching narrows the gap to about 4 times, if your provider's cache rules, expiry and write costs fit your usage. Long prompts also respond more slowly, which matters for anything interactive.
Now compare that with the cost of building RAG. A pilot of $15,000 to $40,000 saves about $0.05 per query against the cached long-context option. That means it takes roughly 300,000 to 800,000 queries to pay back. At 50,000 queries a month, that is 6 to 16 months. At 5,000 queries a month, it is 5 to 13 years. So for a small, stable corpus with low volume, cached long context is the better economic answer. For a large or fast-changing corpus, or high volume, RAG wins. Many production systems use both: retrieve first, then pass a generous set of passages to a long-context model.
Published timelines run from 4 to 8 weeks for a pilot to 16 to 28 weeks for an enterprise platform. Data preparation is the most common source of delay, and one guide suggests adding 2 to 4 weeks when documents are inconsistent or need cleaning. Our software development timeline guide shows how much of any project calendar is waiting rather than coding.
OWASP's 2025 entry on vector and embedding weaknesses lists unauthorized access and data leakage as the first risk. If everyone's documents share one index with no access rules, a question from one user can pull back text from a document they should never see. OWASP recommends permission-aware vector stores and strict logical partitioning for different groups of users. In practice, that means tagging each chunk with who may read it, and filtering retrieval by the asking user's permissions before anything reaches the model.
The same OWASP entry covers data poisoning: someone inserts misleading or instruction-laden text into a document the system will later retrieve. Validate documents before they enter the index, restrict who can add to it, and treat retrieved text as untrusted input to the model.
Patient data brings HIPAA, including business associate agreements with every vendor that touches it. Financial data brings GLBA. State privacy laws add consent and deletion duties, and deleting a source document must also delete its chunks and embeddings. Enterprise customers will ask for evidence such as a SOC 2 report, which our guide to SOC 2 for software startups explains. Log what was asked, what was retrieved and what was answered, with retention rules that match your policies.
Failure | What you see | The fix |
|---|---|---|
Retrieval misses the right passage | Confident but wrong or empty answers | Hybrid search, contextual chunks, reranking, better chunk sizes |
Chunks lose their context | Answers about the wrong company, year or product | Prepend document and section context to each chunk |
Stale index | Answers from old policies | Scheduled re-indexing, version tags, delete-on-change |
Messy source documents | Garbled tables, missing text from scans | Better parsing, OCR, and human cleanup of key documents |
Permission leaks | Users see text they should not | Per-chunk access tags and filtered retrieval |
No evaluation | Nobody knows if a change helped | A test set of real questions, rerun on every change |
Model changes | Quality shifts after a provider update | Pin versions where possible and rerun the test set |
Cost surprises | The monthly bill grows with usage | Cache, trim retrieved context, set usage limits and alerts |
Buy when the use case is standard and the vendor's data connectors cover your sources. Build when your documents, permissions or workflows are specific, or when RAG is part of a product you sell. Blend by using a managed model and vector service and building the ingestion, permissions and evaluation around them, which is where most of the value and risk sit.
If you are weighing partners, our guide to choosing an AI development partner has a scorecard, and the questions below are specific to RAG.
How will you measure retrieval quality and answer quality, and who owns the test set?
How do you enforce who can see which documents?
How do you handle messy PDFs, tables and scanned files?
What happens when a source document is updated or deleted?
Which model providers see our data, and is it ever used for training?
What are the monthly running costs at our expected question volume?
Which roles and how many hours sit behind your quote, and what is excluded?
Who owns the pipeline code, prompts and evaluation data?
We are a good fit when you have a large, changing set of documents, specific permission rules, and want a system integrated into a product or workflow. We are not the right choice if your knowledge base would fit comfortably in a prompt, or if a packaged tool already does the job. We will say so in the first call.
We do not publish a rate card. A quote starts with a one-to-two-week discovery that looks at your documents, your permission rules, your questions and the risks. It produces a roadmap and a scoped estimate tied to roles and hours. We work on a fixed-cost, dedicated-team or staff augmentation basis, depending on how settled your scope is. Our engineers are in-house, and you own the IP.
Start with our RAG application development page, our AI development company in the USA page or our generative AI integration services.
None of these is a RAG deployment, and we will not dress them up as one. They are production AI systems where accuracy, speed and sensitive data mattered.
For M2 Method, a California pelvic health program, we took an AI proof of concept to a production iOS and Android platform with real-time feedback capped at 300 ms. Read the case study.
For Qualis Construction Ltd., we built an AI quantity takeoff application that replaced manual blueprint counting. Read the case study.
We built an AI-powered medical diagnostic system for real-time clinical decision support. Read the case study.
Our Google rating is 4.9 from 126 reviews, and we have delivered 180+ projects across 15+ industries.
RAG is a way of building AI applications so that, before a language model answers, the system searches your own documents and gives the model the most relevant passages to answer from. It lets an assistant use private, current information and cite its sources without retraining the model. Most enterprise AI knowledge assistants use some form of it.
Plan on $15,000 to $40,000 for a single-source pilot, $40,000 to $100,000 for a production system with several sources, permissions and evaluation, and $100,000 to $250,000 or more for enterprise, multi-tenant or regulated builds. Running costs range from about $300 to $13,000 a month. Published guides range from $5,000 to $250,000+, so ask for the roles and hours behind any quote.
Skip RAG when your whole knowledge base fits in roughly 200,000 tokens (about 500 pages) and question volume is low, when questions need exact numbers from structured systems, when a packaged product already does the job, or when answers must be exactly right every time. In those cases, long-context prompting, database queries, a packaged tool or human review may serve you better.
At volume, yes. In our model with $3 per million input tokens, a RAG query costs about $0.0165, a 200,000-token prompt about $0.61, or about $0.066 if fully cached, and a 1-million-token prompt about $3.00. But for a small, stable corpus at low volume, cached long context can beat the cost of building RAG. Replace the assumptions with your provider's prices.
Published timelines run from about 4 to 8 weeks for a pilot, 10 to 16 weeks for a production system and 16 to 28 weeks for an enterprise platform. Data preparation is the most common source of delay, and messy or scanned documents can add 2 to 4 weeks. A one-to-two-week discovery phase helps surface these issues early.
Start with pgvector if you already run PostgreSQL. It costs little and one guide reports it handles up to about 5 million vectors comfortably. Managed services such as Pinecone make sense for larger scale or when you want less operational work, with a $50 monthly minimum on its Standard plan. Real bills can run well above calculator estimates, so budget a margin.
Use permission-aware retrieval. Tag each chunk with who may read it, filter retrieval by the asking user's permissions before anything reaches the model, and partition data between groups or tenants. OWASP's 2025 LLM Top 10 lists unauthorized access and data leakage as the first risk in vector and embedding weaknesses. Also validate documents before indexing and log every retrieval.
Build a test set of real questions with known answers before launch and rerun it after every change. Measure whether retrieved passages are relevant, whether the answer is supported by them, and whether it addresses the question. Track the retrieval failure rate, since Anthropic's tests showed it falling from 5.7% to 1.9% with contextual chunks, keyword search and reranking.
If you have a document set and a question you want answered, book a technical consultation. Bring a sample of the documents, the questions people ask and any rules about who can see what. We will tell you what we would ask in discovery, whether RAG, long context or a packaged tool fits, and where the risk sits, whether or not you end up working with us.
Subscribe to the Akoode newsletter for carefully curated insights on AI, digital intelligence, and real-world innovation. Just perspectives that help you think, plan, and build better.