RAG and LLMs: Building Answers Grounded in Your Data
A practical introduction to RAG and large language models: retrieval, source grounding, permissions, evaluation, and Arabic–English search.
AI / الذكاء الاصطناعي
Build a traceable RAG ingestion pipeline for PDFs: preserve layout, page provenance, meaningful chunks, access metadata, and visible extraction failures.

العربية؟ اقرأ النسخة العربية.
A retrieval-augmented generation system can only retrieve what its ingestion pipeline preserved. A PDF may contain selectable text, scanned pages, multi-column layouts, tables, footnotes, or a mixture. Turning every file into one long text string can erase the structure that helps an answer point back to evidence.
A useful pipeline treats extraction, chunking, indexing, and provenance as connected stages. Each stage should leave enough information to explain what happened to a source and where a retrieved passage came from.
PDF or office file
→ identify version, owner, and access policy
→ extract text and layout, including OCR where needed
→ normalize while retaining page and section provenance
→ create structure-aware chunks
→ attach metadata and access filters
→ embed and index for semantic and exact-term retrieval
→ verify the indexed version and record failures
The exact services can vary. The important design choice is to keep the original source and the extracted representation connected rather than treating the vector index as the only copy.
When a file enters the pipeline, record a stable document ID, source URI, tenant or collection, version, content hash, received time, and access policy. Keep the source's display name and canonical location for citations. If a document is replaced, retain a relationship to the prior version so an answer can distinguish “current policy” from an older revision.
A content hash helps an ingestion job recognize an unchanged file. It does not replace authorization: the application must still enforce the caller's access to the document and its passages at retrieval time.
Text-based PDFs can often be parsed directly. Scans and image-only pages need OCR. Mixed files may require both paths. For tables, preserve rows, columns, headers, and captions when possible; flattening a table into unrelated lines can make values ambiguous.
Microsoft's Document Intelligence RAG guidance describes using the Layout model's Markdown output to split around paragraphs and create table-aware chunks. The Layout model documentation explains extraction of text, tables, selection marks, and document structure.
Preserve the page number and, where the extractor provides it, the source span or bounding region for each extracted element. Store OCR confidence or extraction warnings as quality signals, not as a guarantee that the text is correct. Keep the source file so a reviewer can inspect a suspicious passage.
Remove repeated headers and footers only when they are genuinely boilerplate. Preserve section headings, list boundaries, table captions, and page transitions that matter to meaning. Do not silently join a sentence split across columns or pages. A normalization step should produce both readable text and links back to the source elements used to create it.
A practical chunk record might contain:
{
"documentId": "policy-42",
"documentVersion": "2026-11",
"sectionPath": ["Exports", "Retention"],
"pageStart": 4,
"pageEnd": 5,
"sourceUri": "https://docs.example/policy-42",
"accessScope": "team-a",
"text": "The exported report is available for seven days."
}
The values above are illustrative. In production, IDs and access rules should come from the source system, not from the model that writes an answer.
Chunking keeps passages small enough for embedding and answer generation, but a fixed character count can split a heading from its explanation or a table from its caption. Start with headings and paragraph boundaries, then split oversized sections using a token budget. Keep a table together when it is small enough; for a large table, make row groups self-describing by repeating relevant column headings.
The Azure AI Search chunking guide explains why chunking can help when documents exceed model inputs or when a whole document is poorly represented by one vector. There is no universal chunk size or overlap that fits every corpus: evaluate the choice on representative questions and document types.
Attach the section title and compact document context to each chunk. Anthropic's Contextual Retrieval article explores adding chunk-specific context before embedding and lexical indexing. Treat that as a technique to evaluate for passages that lose their subject when separated from surrounding text, not as a substitute for preserving the original.
An embedding maps text to a vector used to compare semantic relatedness. OpenAI's embeddings guide describes creating vectors from text and notes that model choice determines vector length. Keep the embedding model and its configuration with the index version so an index can be rebuilt consistently.
Vector similarity is useful for paraphrases, but support questions often contain exact IDs, error codes, product names, or version strings. A lexical path such as BM25 can complement vector retrieval. A hybrid query combines candidate sets and ranks them before the application selects evidence for the model.
Keep source ID, version, section, page, canonical URL, and access fields on the indexed record. Before returning a passage, apply the user's permitted scope and filter out superseded or revoked content. A relevant vector must never bypass document authorization.
A robust ingestion job records a status for each stage: received, extracted, chunked, embedded, indexed, or failed. Give a versioned source an idempotent ingestion key so a retry updates the same logical records instead of creating duplicates. If extraction fails on one page, keep the failure and source location visible instead of quietly indexing incomplete text as complete.
async function ingest(source) {
const manifest = await registerSource(source);
const layout = await extractLayout(source);
const chunks = await splitByStructure(layout, { tokenBudget: 700 });
const records = chunks.map((chunk) => ({
id: source.id + ":" + source.version + ":" + chunk.index,
documentId: source.id,
documentVersion: source.version,
sectionPath: chunk.sectionPath,
pageStart: chunk.pageStart,
pageEnd: chunk.pageEnd,
sourceUri: manifest.canonicalUri,
text: chunk.text
}));
await embedAndUpsert(records);
await verifyIndexedVersion(source.id, source.version);
}
The token budget in this sketch is an example, not a universal recommendation. A real system should use the selected model's current input limits, reserve room for metadata, and test table-heavy and multilingual documents.
Use a small, reviewed set of questions with known supporting pages. Ask whether extraction preserved the needed sentence or table row, whether the right chunk was retrieved, whether the cited page matches the source, and whether a stale version or restricted passage was excluded. Track failure categories separately: OCR error, bad reading order, chunk boundary, missing metadata, ranking, or access filter.
If a question's evidence is absent, the answer layer should say so or request clarification. Good parsing reduces avoidable retrieval problems, but it cannot guarantee every generated answer is correct.
Want the Arabic edition? اقرأ النسخة العربية.
YOUR TAKE
Choose one reaction for this article.
A cookie helps limit repeat responses. Your feedback is private.