RAG Development for US Businesses: Architecture, Cost and When Not to Use It in 2027

RAG Development for US Businesses: Architecture, Cost and When Not to Use It in 2027

What is RAG?

Retrieval-augmented generation (RAG) is a way of building AI applications so that, before a language model answers a question, the system first searches your own documents and gives the model the most relevant passages to answer from. It lets an AI assistant use private, current information and cite its sources without retraining the model.

Short answer

A production RAG system for a US business typically costs $15,000 to $40,000 as a single-source pilot, $40,000 to $100,000 for a production system with several sources, permissions and evaluation, and $100,000 to $250,000 or more for an enterprise, multi-tenant or regulated build. Running costs range from a few hundred dollars to several thousand dollars a month. And RAG is not always the right answer: if your whole knowledge base fits in about 200,000 tokens (roughly 500 pages) and gets few questions, putting it straight into the model's prompt can be cheaper and simpler.

Almost every company with a pile of documents has had the same idea: "let people ask questions and get answers from our own files." Vendors call it RAG, and some call it enterprise search or a knowledge assistant. It is a good idea, and it is also one of the easiest AI projects to build badly. The demo works on ten documents. Then it meets ten thousand, with outdated versions, scanned PDFs and files that only some employees should see.

This guide is for the person responsible for getting it to work: a CTO, a head of operations, a product owner. We cover when RAG is the right tool and when it is not, the architecture that holds up in production, what it costs with an original per-query cost model, the security issue most demos ignore, and the failure modes to plan for.

We build AI systems for a living, so we have a point of view. Where the cheaper answer is not to build RAG at all, we say so.

What RAG is and how it works

The five steps

  1. Ingest. Your documents are collected, cleaned and split into passages called chunks.

  2. Index. Each chunk is converted into a numeric representation (an embedding) and stored in a search index, often a vector database.

  3. Retrieve. When a user asks a question, the system searches the index and finds the most relevant chunks.

  4. Generate. The question and the retrieved chunks go to a language model, which writes the answer.

  5. Cite and check. The system shows which passages the answer came from, so a person can verify it.

OWASP's LLM Top 10 project lead has said that some form of RAG is now the default architecture for enterprise language model applications, and the project added a dedicated entry on vector and embedding weaknesses to its 2025 Top 10.

RAG compared with the alternatives

Approach

What it does

Best for

RAG

Searches your documents at question time and gives the model the relevant passages

Large, changing, private knowledge bases that need source citations

Long-context prompting

Puts the whole document set into the prompt

Small, stable document sets, or deep reasoning across a few long documents

Fine-tuning

Adjusts the model itself with your examples

Style, format and behavior, not facts that change

Traditional search

Returns a list of matching documents

When users want to read the documents themselves

SQL or API queries

Retrieves exact records from structured systems

Numbers, balances, inventory and anything that must be exact

When not to use RAG

Your knowledge base is small

Anthropic's own guidance in its Contextual Retrieval write-up is that if your knowledge base is smaller than about 200,000 tokens, roughly 500 pages, you can simply include the whole thing in the prompt, ideally with prompt caching. Frontier models now accept far more than that: a June 2026 research paper notes that OpenAI's GPT-5.5 supports a context window of about 1.05 million tokens, and frames the choice between long context and RAG as a trade-off between accuracy and the recurring token cost of reading more. The cost model below shows how big that cost gap is.

Your question needs exact numbers

RAG finds passages of text. It is a poor tool for "what was our revenue in Q3" or "how many units are in stock," which belong to a database query. Use an AI layer that writes and runs a query against structured systems, and keep RAG for the unstructured material around it.

A packaged product already does the job

If the use case is standard, such as customer support or an internal helpdesk, a packaged product may do it for a published per-resolution price. Our guides to custom software vs SaaS and AI agent development for US businesses cover the build-or-buy math.

The answers must be exactly right every time

RAG reduces hallucination and does not eliminate it. If a wrong answer causes harm, such as a medical dosage or a legal clause, the system needs human review or a deterministic path, with RAG as a helper rather than the decision-maker.

A quick decision guide

Situation

Usually the better choice

Under about 200,000 tokens of stable content, low question volume

Long-context prompting with caching

Hundreds of thousands to millions of tokens, content changes often

RAG

Questions about numbers in structured systems

SQL or API queries, with AI as the interface

Standard support or helpdesk use case

A packaged product first

Deep reasoning across a few long documents

Long context, possibly with retrieval to narrow the set

Strict latency target of about two seconds

RAG

The architecture that holds up in production

Ingestion and chunking

Most RAG projects succeed or fail here. Messy PDFs, scans, tables and slide decks need parsing that preserves structure, and chunks must be sized so that each carries enough meaning to be found and understood. One 2026 cost guide estimates that data preparation and ingestion are 40% to 50% of the total engineering work, which matches what teams discover the hard way.

Hybrid retrieval

Embeddings find passages with similar meaning and can miss an exact term such as an error code or a policy number. Keyword search (BM25) catches those. Production systems combine both and merge the results.

Contextual chunks and reranking

A chunk such as "revenue grew 3% over the previous quarter" is useless without knowing which company and which quarter. Anthropic's Contextual Retrieval method prepends a short explanatory context to each chunk before indexing. In its tests, the top-20 retrieval failure rate fell from 5.7% to 3.7% with contextual embeddings, to 2.9% when combined with keyword search (a 49% reduction), and to 1.9% when a reranking step was added (a 67% reduction). Those are one company's results on its own test sets, so measure on your own data, but the order of the gains is a useful guide to where effort pays.

Generation with citations

Instruct the model to answer only from the retrieved passages, to say when it cannot find an answer, and to cite its sources. Showing the source passage is the cheapest trust mechanism you have.

Permission-aware retrieval

The security section below covers this in detail, but design it in from day one. Retrieval must respect who is allowed to see what.

Evaluation

Measure three things: whether the retrieved passages are relevant to the question, whether the answer is supported by those passages, and whether the answer addresses the question. Build a test set of real questions with known answers before launch, and rerun it after every change. A system with no evaluation set is a system nobody can improve.

Monitoring and upkeep

Log questions, retrieved sources and user feedback. Watch for stale documents, questions the system cannot answer and drift after model changes. Index freshness is an operations job, not a one-time setup.

What RAG development costs in the USA: planning bands for 2027

Our planning bands

These are our planning figures, built from the published ranges in the next section and sized for the US market. They are not a quote.

Tier

Build cost

Typical timeline

Monthly running cost

What it usually includes

Single-source pilot

$15,000 to $40,000

4 to 8 weeks

$300 to $800

One document source, basic chunking, vector search, a chat interface, citations.

Production system

$40,000 to $100,000

10 to 16 weeks

$1,000 to $3,000

Several sources, hybrid retrieval and reranking, role-based access, audit logging, an evaluation set, integrations.

Enterprise, multi-tenant or regulated

$100,000 to $250,000+

16 to 28 weeks

$2,000 to $13,000

Large corpora, strict permissions, compliance work, agent workflows, on-premises or private cloud options.

Why published prices disagree

Source (2025 to 2026 guides)

What it says

ScalaCode

$10,000 for a basic prototype to $150,000 or more for an enterprise agentic system. Vector database from $0 (pgvector) to $2,000+ a month.

RaftLabs

A production RAG system costs $30,000 to $60,000 to build, with $300 to $2,500 a month in infrastructure. Simple pipelines take 4 to 6 weeks and enterprise platforms 14 to 22 weeks.

Ortem

Simple RAG $15,000 to $25,000, production $40,000 to $80,000, enterprise on-premises $80,000 to $150,000+, with timelines from 4 weeks to 28 weeks.

Dyyota

Enterprise RAG $40,000 to $150,000+ with $2,000 to $8,000 a month in ongoing costs.

Huzaifa Tahir

$5,000 prototypes to $250,000+ enterprise platforms, with monthly operations from $200 to $13,000.

Three things explain the spread. The guides define "RAG" differently, so a prototype on one document source shares a price list with a multi-tenant platform. They differ on whether evaluation, permissions and data cleaning are included. And nearly all are written by firms that sell this work. Use them to find the range, then test any quote against hours and roles, as we show in our custom software development cost guide.

Where the money really goes

  • Data preparation. Cleaning, parsing and structuring documents is the largest single piece of engineering effort.

  • Evaluation. Building the test set and the measurement pipeline is the line item vendors forget to quote.

  • Permissions and audit. Mirroring your access rules into search takes real design work.

  • The vector database, less than people think. pgvector on an existing Postgres database costs little and one guide reports it handles up to about 5 million vectors comfortably. Managed services such as Pinecone start at a $50 monthly minimum on the Standard plan and can reach thousands at large scale. One vendor reports that real vector database bills run 2.5 to 4 times the pricing calculator estimate once re-indexing and operations are included, so treat that as a vendor claim and budget a margin.

  • Model usage. It scales with questions asked, which the next section models.

The per-query cost model: RAG vs long context

Here is a simple model of what one question costs. Every input is an assumption you should replace with your provider's current prices: $3 per million input tokens and $15 per million output tokens, typical of mid-tier models in 2025 and 2026. A RAG query sends about 3,500 input tokens (five retrieved chunks plus the question and instructions) and gets back 400 tokens. A long-context query sends the whole document set.

Approach

Cost per query

5,000 queries a month

50,000 queries a month

RAG (3,500 tokens in)

$0.0165

$83

$825

Long context, 200,000 tokens

$0.606

$3,030

$30,300

Long context, 200,000 tokens, fully cached at 10% of input price

$0.066

$330

$3,300

Long context, 1 million tokens

$3.006

$15,030

$150,300

What the model says

Uncached, a 200,000-token prompt costs about 37 times more per query than RAG, and a 1-million-token prompt about 180 times more. Caching narrows the gap to about 4 times, if your provider's cache rules, expiry and write costs fit your usage. Long prompts also respond more slowly, which matters for anything interactive.

Now compare that with the cost of building RAG. A pilot of $15,000 to $40,000 saves about $0.05 per query against the cached long-context option. That means it takes roughly 300,000 to 800,000 queries to pay back. At 50,000 queries a month, that is 6 to 16 months. At 5,000 queries a month, it is 5 to 13 years. So for a small, stable corpus with low volume, cached long context is the better economic answer. For a large or fast-changing corpus, or high volume, RAG wins. Many production systems use both: retrieve first, then pass a generous set of passages to a long-context model.

Timeline: how long RAG takes

Published timelines run from 4 to 8 weeks for a pilot to 16 to 28 weeks for an enterprise platform. Data preparation is the most common source of delay, and one guide suggests adding 2 to 4 weeks when documents are inconsistent or need cleaning. Our software development timeline guide shows how much of any project calendar is waiting rather than coding.

Security and compliance for US deployments

Permission-aware retrieval

OWASP's 2025 entry on vector and embedding weaknesses lists unauthorized access and data leakage as the first risk. If everyone's documents share one index with no access rules, a question from one user can pull back text from a document they should never see. OWASP recommends permission-aware vector stores and strict logical partitioning for different groups of users. In practice, that means tagging each chunk with who may read it, and filtering retrieval by the asking user's permissions before anything reaches the model.

Poisoned or malicious documents

The same OWASP entry covers data poisoning: someone inserts misleading or instruction-laden text into a document the system will later retrieve. Validate documents before they enter the index, restrict who can add to it, and treat retrieved text as untrusted input to the model.

Sensitive data and compliance

Patient data brings HIPAA, including business associate agreements with every vendor that touches it. Financial data brings GLBA. State privacy laws add consent and deletion duties, and deleting a source document must also delete its chunks and embeddings. Enterprise customers will ask for evidence such as a SOC 2 report, which our guide to SOC 2 for software startups explains. Log what was asked, what was retrieved and what was answered, with retention rules that match your policies.

Common failure modes and how to fix them

Failure

What you see

The fix

Retrieval misses the right passage

Confident but wrong or empty answers

Hybrid search, contextual chunks, reranking, better chunk sizes

Chunks lose their context

Answers about the wrong company, year or product

Prepend document and section context to each chunk

Stale index

Answers from old policies

Scheduled re-indexing, version tags, delete-on-change

Messy source documents

Garbled tables, missing text from scans

Better parsing, OCR, and human cleanup of key documents

Permission leaks

Users see text they should not

Per-chunk access tags and filtered retrieval

No evaluation

Nobody knows if a change helped

A test set of real questions, rerun on every change

Model changes

Quality shifts after a provider update

Pin versions where possible and rerun the test set

Cost surprises

The monthly bill grows with usage

Cache, trim retrieved context, set usage limits and alerts

Build, buy or blend

Buy when the use case is standard and the vendor's data connectors cover your sources. Build when your documents, permissions or workflows are specific, or when RAG is part of a product you sell. Blend by using a managed model and vector service and building the ingestion, permissions and evaluation around them, which is where most of the value and risk sit.

If you are weighing partners, our guide to choosing an AI development partner has a scorecard, and the questions below are specific to RAG.

Questions to ask a RAG partner

  • How will you measure retrieval quality and answer quality, and who owns the test set?

  • How do you enforce who can see which documents?

  • How do you handle messy PDFs, tables and scanned files?

  • What happens when a source document is updated or deleted?

  • Which model providers see our data, and is it ever used for training?

  • What are the monthly running costs at our expected question volume?

  • Which roles and how many hours sit behind your quote, and what is excluded?

  • Who owns the pipeline code, prompts and evaluation data?

How Akoode approaches RAG projects

When we are the right choice, and when we are not

We are a good fit when you have a large, changing set of documents, specific permission rules, and want a system integrated into a product or workflow. We are not the right choice if your knowledge base would fit comfortably in a prompt, or if a packaged tool already does the job. We will say so in the first call.

How we scope and price

We do not publish a rate card. A quote starts with a one-to-two-week discovery that looks at your documents, your permission rules, your questions and the risks. It produces a roadmap and a scoped estimate tied to roles and hours. We work on a fixed-cost, dedicated-team or staff augmentation basis, depending on how settled your scope is. Our engineers are in-house, and you own the IP.

Start with our RAG application development page, our AI development company in the USA page or our generative AI integration services.

Proof from delivered projects

None of these is a RAG deployment, and we will not dress them up as one. They are production AI systems where accuracy, speed and sensitive data mattered.

  • For M2 Method, a California pelvic health program, we took an AI proof of concept to a production iOS and Android platform with real-time feedback capped at 300 ms. Read the case study.

  • For Qualis Construction Ltd., we built an AI quantity takeoff application that replaced manual blueprint counting. Read the case study.

  • We built an AI-powered medical diagnostic system for real-time clinical decision support. Read the case study.

Our Google rating is 4.9 from 126 reviews, and we have delivered 180+ projects across 15+ industries.

Frequently asked questions

What is RAG (retrieval-augmented generation)?

RAG is a way of building AI applications so that, before a language model answers, the system searches your own documents and gives the model the most relevant passages to answer from. It lets an assistant use private, current information and cite its sources without retraining the model. Most enterprise AI knowledge assistants use some form of it.

How much does RAG development cost in the USA in 2027?

Plan on $15,000 to $40,000 for a single-source pilot, $40,000 to $100,000 for a production system with several sources, permissions and evaluation, and $100,000 to $250,000 or more for enterprise, multi-tenant or regulated builds. Running costs range from about $300 to $13,000 a month. Published guides range from $5,000 to $250,000+, so ask for the roles and hours behind any quote.

When should I not use RAG?

Skip RAG when your whole knowledge base fits in roughly 200,000 tokens (about 500 pages) and question volume is low, when questions need exact numbers from structured systems, when a packaged product already does the job, or when answers must be exactly right every time. In those cases, long-context prompting, database queries, a packaged tool or human review may serve you better.

Is RAG cheaper than long-context prompting?

At volume, yes. In our model with $3 per million input tokens, a RAG query costs about $0.0165, a 200,000-token prompt about $0.61, or about $0.066 if fully cached, and a 1-million-token prompt about $3.00. But for a small, stable corpus at low volume, cached long context can beat the cost of building RAG. Replace the assumptions with your provider's prices.

How long does it take to build a RAG system?

Published timelines run from about 4 to 8 weeks for a pilot, 10 to 16 weeks for a production system and 16 to 28 weeks for an enterprise platform. Data preparation is the most common source of delay, and messy or scanned documents can add 2 to 4 weeks. A one-to-two-week discovery phase helps surface these issues early.

Which vector database should I use for RAG?

Start with pgvector if you already run PostgreSQL. It costs little and one guide reports it handles up to about 5 million vectors comfortably. Managed services such as Pinecone make sense for larger scale or when you want less operational work, with a $50 monthly minimum on its Standard plan. Real bills can run well above calculator estimates, so budget a margin.

How do I keep RAG from leaking confidential documents?

Use permission-aware retrieval. Tag each chunk with who may read it, filter retrieval by the asking user's permissions before anything reaches the model, and partition data between groups or tenants. OWASP's 2025 LLM Top 10 lists unauthorized access and data leakage as the first risk in vector and embedding weaknesses. Also validate documents before indexing and log every retrieval.

How do I measure whether my RAG system works?

Build a test set of real questions with known answers before launch and rerun it after every change. Measure whether retrieved passages are relevant, whether the answer is supported by them, and whether it addresses the question. Track the retrieval failure rate, since Anthropic's tests showed it falling from 5.7% to 1.9% with contextual chunks, keyword search and reranking.

Check whether RAG fits your documents

If you have a document set and a question you want answered, book a technical consultation. Bring a sample of the documents, the questions people ask and any rules about who can see what. We will tell you what we would ask in discovery, whether RAG, long context or a packaged tool fits, and where the risk sits, whether or not you end up working with us.

Tags
#RAG#EnterpriseAI#AIDevelopment

Get In Touch Now

= ?

Stay Informed with Thoughtful Innovation

Subscribe to the Akoode newsletter for carefully curated insights on AI, digital intelligence, and real-world innovation. Just perspectives that help you think, plan, and build better.