RAG — Grounding AI in Your Documents

🤔 The Problem: LLMs Don't Know Your Hospital

The LLM behind AgentSea was trained on public internet text up to some cut-off date. It has never read:

Your staff handbook
Your procurement policy v3.1
Last week\'s MOH circular
Today\'s vendor proposal
Your meeting minutes from this morning

So if you ask "What\'s our annual leave entitlement?", the model has two bad options: (1) say "I don\'t know" — useless, or (2) make up a plausible-sounding answer based on what other organisations typically have — dangerous hallucination.

📄 Without RAG

You: "How many days of annual leave is a Senior Pharmacist entitled to?"
Plain LLM: "Based on typical Singapore healthcare practice, Senior Pharmacists usually receive 18–21 days of annual leave depending on tenure." → Made up. Could be wrong. Cannot be cited.

RAG (Retrieval-Augmented Generation) solves this: before the LLM answers, we automatically retrieve the most relevant chunks of your documents and put them in the prompt. Now the model has the real source material — and can answer correctly with citations.

📄 With RAG

You: "How many days of annual leave is a Senior Pharmacist entitled to?"
RAG-enabled: [retrieves Section 4.3 of Staff Handbook v3.1] "According to our Staff Handbook (Section 4.3), Senior Pharmacists receive 21 days of annual leave after 3 years of continuous service. Source: Staff Handbook v3.1, p. 24." → Grounded. Citable. Verifiable.

⚖️ RAG vs. Pure LLM

❌ PURE LLM

"Confidently wrong" risk

Hallucinates plausible-but-fabricated answers. No way to cite sources. Knowledge frozen at training cut-off.

"Senior Pharmacists usually receive 18–21 days…"

✅ WITH RAG

Grounded in your sources

Answers come from your actual documents. Citations included. Up-to-date as soon as you upload new files.

"Per Staff Handbook §4.3: 21 days after 3 years."

🎯 Why RAG is the Killer Pattern

🔓

Unlocks Private Data

Your documents stay in your AWS account. The LLM only sees the chunks that get retrieved.

📎

Enables Citations

Every answer can show exactly which document and section it came from. Auditable, defensible.

⚡

No Retraining Needed

Add a new MOH circular today; AgentSea uses it tomorrow. No model retraining. Just upload.

🛡️

Reduces Hallucination

When you give the model the source text, it has far less reason to invent. Grounding works.

🔗 The 5-Step RAG Pipeline

Here\'s what happens when you ask AgentSea a question that requires document grounding:

📝

Step 1 · Embed your question

When you type "What\'s our annual leave entitlement for senior pharmacists?", AgentSea sends the question to an embedding model that turns it into a 1,536-dimension vector. This vector captures the meaning of your question.

✂️ Chunking — Splitting Your Documents

Before any of this works, your documents have to be split into chunks and embedded. You can\'t embed a 200-page handbook as one vector — you\'d lose all the local detail. So we cut it into smaller pieces.

Naive: Split by character count

The simplest approach: every 1,000 characters becomes one chunk. Easy to implement, but breaks paragraphs and sentences mid-thought:

Chunk 1

"...annual leave is calculated based on continuous service. Junior staff receive 14 days while sen

Chunk 2

ior pharmacists receive 21 days after 3 years. Maternity leave entitlements are detailed in Sect

Chunk 3

ion 4.7 below. All leave applications must be submitted at least 14 days in advance via the HR p

⚠️

The problem: Chunk 1 ends mid-word ("sen-"). Chunk 2 starts with the rest. If you ask about "Senior Pharmacist leave", the search may not match either chunk because the meaningful phrase is split.

Better: Split by structure

📄 By section heading

Each subsection (e.g., 4.3 Annual Leave) becomes one chunk. Preserves logical units. Best for structured policy documents.

¶ By paragraph

Each paragraph = one chunk. Simple and reasonably semantic. Good for general policy text.

💬 By sentence

Most granular. Each sentence stands alone. Best when answers are typically a single fact ("21 days").

🔄 With overlap

Each chunk overlaps the previous by ~10–20%. Prevents critical info being split. Standard practice.

💡

For Synapxe documents: Section-heading-based chunking with 10% overlap works well for most policy documents and MOH circulars. Aim for chunks of 200–500 words each. Too small = loses context; too big = retrieval becomes imprecise.

🎯 Retrieval — Finding the Right Chunks

Once your documents are chunked and embedded, retrieval is straightforward:

Embed the user\'s question into a vector
Compute cosine similarity between question vector and every chunk vector
Sort chunks by similarity score (descending)
Return the top K (typically K = 3 to 5)

Worked example — query against your handbook

🔍

"How many days of annual leave does a senior pharmacist get?"

0.91

📄 Handbook §4.3

"Annual leave entitlements: Senior Pharmacists receive 21 days after 3 years of continuous service..."

0.78

📄 Handbook §4.7

"Leave application process: Submit through HR portal at least 14 days in advance..."

0.34

📄 §6.1

"Performance evaluations are conducted annually by direct supervisors..."

0.18

📄 §8.2

"IT equipment requests must be approved through the procurement channel..."

The top 2 chunks (with similarity > 0.7) are passed to the LLM. Lower-scoring chunks are ignored.

💡

The trade-off in K (number of chunks retrieved): Too few (K=1) and you might miss the right chunk. Too many (K=20) and you waste tokens on irrelevant material — and the LLM may get confused by noise. Most production systems use K = 3 to 5 with a relevance threshold (e.g., similarity > 0.7).

📋 Prompt Assembly — How the Final Prompt is Built

After retrieval, AgentSea builds the actual prompt sent to the LLM. It has 3 parts: system instructions, retrieved context, and your question.

SYSTEM PROMPT (always present)

You are an AgentSea assistant for Synapxe Healthcare admin staff. Answer using ONLY the provided context. If the context does not contain the answer, say "I cannot find this in the provided documents." Always cite the source section.

RETRIEVED CONTEXT (top-K chunks)

[Source: Staff Handbook v3.1, §4.3]
"Annual leave entitlements: Junior staff receive 14 days. Senior Pharmacists receive 21 days after 3 years of continuous service. Department heads may grant up to 5 additional discretionary days per year for exceptional circumstances..."

[Source: Staff Handbook v3.1, §4.7]
"Leave application process: All leave must be submitted through the HR portal at least 14 days in advance, except in cases of medical emergency where retrospective approval is permitted..."

USER QUESTION

How many days of annual leave does a Senior Pharmacist get?

What the LLM sees vs what you typed

You typed one short question. The LLM receives a much longer prompt with all the source material baked in. This is the key insight — you don\'t have to know which policy section is relevant. RAG figures it out automatically.

💡

Why citations matter for healthcare: The system prompt tells the LLM to cite. The retrieved chunks include source markers. So the answer can include "Per Staff Handbook §4.3:" — making the response auditable. For compliance and HR work, this is non-negotiable.

📊 Token Budget Math

RAG isn\'t free. Every chunk you add to the prompt costs tokens, which costs both money and context-window space:

Prompt component	Typical tokens	What it covers
System prompt	~150	Persona, rules, output format
Retrieved chunks (K=3, ~400 tokens each)	~1,200	The relevant document content
User question	~30	Whatever you typed
Reserved for response	~500	The model\'s answer
Total	~1,880	Per query

For a model with 128K context window, you have lots of headroom. For older 4K-window models, RAG forces tight chunking. AgentSea uses Bedrock\'s modern models — plenty of room.

🌐 RAG in AgentSea — What\'s Already Live

RAG isn\'t a future feature. It\'s already powering several AgentSea capabilities you\'re using today:

📄

Document Analysis

Upload a vendor proposal, ask questions, get answers grounded in that document. Single-doc RAG.

📚

Knowledge Spaces (Aug 2026)

Phase 1B brings persistent multi-document RAG: pre-load your entire policy library, query it any time.

🔗

SharePoint Integration (Aug 2026)

Auto-index your SharePoint sites. Updates daily. Always queries against the latest version.

💬

Conversational Q&A

Ask follow-up questions. AgentSea retains conversation context and re-retrieves chunks as needed.

Your role as a user

For RAG to work well for you, three things matter:

Upload the right documents — RAG can only retrieve from what you\'ve given it. Garbage in, garbage out.
Ask specific questions — "What\'s the leave policy?" is too vague. "What\'s the annual leave entitlement for Senior Pharmacists with 3+ years of service?" retrieves much better.
Verify citations — Always check the source citation matches what you\'d expect. The model can occasionally still hallucinate even with RAG.

⚠️ When RAG Falls Short

RAG is brilliant for fact retrieval but has real limitations. Know when not to rely on it:

Limitation	Why it happens	Mitigation
Spans multiple chunks	Answer needs info from chunks 3, 8, and 14 — but only top 5 retrieved	Larger K · semantic chunking · re-ranking
Numerical aggregation	"Total spending across all 12 vendors" requires reading all docs	Use a calculator/database tool, not RAG alone
"Latest version" ambiguity	Retrieved chunk could be from old policy v2 instead of current v3	Strict version metadata in chunks · filter by version
Out-of-scope queries	Asking about something not in any document	System prompt: "say I don\'t know" · relevance threshold

✅ Key Takeaways

RAG = retrieve relevant chunks first, then ask the LLM. Grounding via context.
5 steps: embed query → search → retrieve top-K → augment prompt → generate.
Chunking matters — section-based, ~200–500 words, with 10% overlap.
Cosine similarity ranks chunks by relevance. Typically retrieve top 3–5.
Always include system prompt rules requiring citations and "say I don\'t know" if missing.
RAG enables AgentSea to answer from your documents — not generic internet knowledge.

⏭️ Up Next: Prompt Engineering

You now understand the full machine: tokens, embeddings, transformer, RAG. The last skill is writing prompts that produce reliable output — the capstone explainer.

✍️ Continue to Prompt Engineering →