LlamaIndex — RAG Framework Complete Guide

In this tutorial, you'll learn about LlamaIndex. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

LlamaIndex is a data framework for building RAG applications that connect LLMs to your own data, enabling semantic search, document Q&A, and knowledge-augmented AI systems without training custom models.

What You'll Learn

• How to ingest documents (PDFs, web pages, databases) into LlamaIndex
• Building vector indexes for semantic search
• Creating query engines with retrieval-augmented generation
• Advanced RAG patterns: hierarchical indexes, routing, and agents

Why LlamaIndex Matters

LLMs know only what they were trained on. For questions about your own documents, codebase, or database, you need RAG. LlamaIndex handles the entire pipeline: document Parsing, chunking, embedding, storing, retrieving, and augmenting LLM prompts. It's the most popular dedicated RAG framework, used by 500K+ developers.

Durga Antivirus Pro uses LlamaIndex-powered RAG to query threat intelligence databases. Doda Browser employs it for local bookmark and history search.

Learning Path

flowchart LR
  A[OpenAI API] --> B[LangChain]
  B --> C[CrewAI]
  C --> D[LlamaIndex
You are here]
  D --> E[Hugging Face]
  style D fill:#dbeafe,stroke:#2563eb

RAG Explained

Retrieval-Augmented Generation works in two phases:

1. Indexing: Your documents are split into chunks, converted to vector embeddings, and stored in a vector database
2. Querying: When a user asks a question, the system finds relevant chunks by semantic similarity and passes them to the LLM as context

Think of it as giving the LLM an open-book exam. Instead of relying on memory (training data), it can look up the answer in your documents.

flowchart LR
    subgraph Indexing
        A[Documents] --> B[Chunking]
        B --> C[Embedding]
        C --> D[Vector Store]
    end
    subgraph Querying
        E[User Query] --> F[Query Embedding]
        F --> G[Similarity Search]
        D --> G
        G --> H[Retrieved Chunks]
        H --> I[LLM + Context]
        I --> J[Answer]
    end
    style D fill:#fef3c7,stroke:#d97706
    style I fill:#dbeafe,stroke:#2563eb

Getting Started

Installation

pip install llama-index llama-index-embeddings-openai llama-index-llms-openai

Basic Indexing

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(model="gpt-4")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

# Load documents from a directory
documents = SimpleDirectoryReader("./data").load_data()

# Create the index
index = VectorStoreIndex.from_documents(documents)

# Persist the index
index.storage_context.persist(persist_dir="./storage")

print(f"Indexed {len(documents)} documents")

Expected output:

Indexed 3 documents

This creates embeddings for every document in the ./data directory and stores them in a persistent vector index under ./storage.

Basic Querying

from llama_index.core import StorageContext, load_index_from_storage

# Rebuild storage context
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)

# Create a query engine
query_engine = index.as_query_engine()

response = query_engine.query("What are the key features of Doda Browser?")
print(response)

Expected output: A detailed answer about Doda Browser's features, extracted from the indexed documents, with citations to source chunks.

Advanced Indexing Strategies

Document Chunking

The chunk size and overlap significantly affect retrieval quality:

from llama_index.core.node_parser import SentenceSplitter

parser = SentenceSplitter(
    chunk_size=512,      # Tokens per chunk
    chunk_overlap=50,    # Overlap between chunks
    separator=" ",       # Word boundary separator
)

nodes = parser.get_nodes_from_documents(documents)
index = VectorStoreIndex(nodes)
print(f"Documents: {len(documents)} → Nodes/chunks: {len(nodes)}")

Expected output:

Documents: 3 → Nodes/chunks: 24

Why chunk size matters: Small chunks (128-256 tokens) improve precision but may miss context. Large chunks (1024+ tokens) provide more context but reduce retrieval accuracy. 512 tokens is a good starting point.

Metadata Filters

Add metadata to documents for filtered retrieval:

from llama_index.core import Document

doc_with_meta = Document(
    text="Doda Browser compresses pages for faster loading.",
    metadata={
        "product": "Doda Browser",
        "category": "performance",
        "version": "5.2"
    }
)

index = VectorStoreIndex.from_documents([doc_with_meta])

# Query with metadata filter
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

filters = MetadataFilters(
    filters=[ExactMatchFilter(key="product", value="Doda Browser")]
)
query_engine = index.as_query_engine(filters=filters)

Expected output: Queries only return results where the product metadata matches "Doda Browser".

Query Engine Types

LlamaIndex offers multiple query engine strategies:

Router Query Engine

Route questions to different indexes based on topic:

from llama_index.core.query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector

# Create specialized indexes
browser_index = VectorStoreIndex.from_documents(browser_docs)
security_index = VectorStoreIndex.from_documents(security_docs)

router = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),
    query_engine_tools=[
        browser_index.as_query_engine(),
        security_index.as_query_engine(),
    ]
)

response = router.query("How does Doda Browser protect my privacy?")

Expected output: The router identifies this as a browser-related question and routes it to the browser index. A security question routes to the security index.

Common Errors

1. Embedding Mismatch

Using a different embedding model at query time than at index time produces garbage results. Always verify embed_model is consistent between indexing and querying.

2. Chunk Boundary Issues

A question's answer may be split across two chunks. Use chunk_overlap=50-100 tokens to ensure context isn't cut off at boundaries.

3. Not Persisting the Index

Building the index is expensive. Always call storage_context.persist() and use load_index_from_storage() to avoid re-indexing every time.

4. Ignoring Token Limits

Retrieved chunks + query + system prompt must fit the LLM's context window. Set similarity_top_k=3 (not the default 10) for large documents with 8K context models.

5. Using the Wrong Chunk Size

Code files need smaller chunks (256 tokens) than prose documents (512-1024). PDFs with tables need custom Parsing to preserve structure.

6. Missing Document Parsers

LlamaIndex has specialized readers for PDFs, Notion, Confluence, databases, and more. Using SimpleDirectoryReader on a PDF extracts raw text without layout — use PDFReader for better results.

Practice Questions

1. What is the purpose of embedding models in LlamaIndex?
   They convert text chunks into vector representations for semantic similarity search during retrieval.

2. Why do you need to persist the index?
   Building the index is computationally expensive. Persisting saves it to disk so you can reload it without re-processing all documents.

3. What is the trade-off between small and large chunk sizes?
   Small chunks improve retrieval precision but may lose context. Large chunks provide context but reduce the relevance of retrieved results.

4. How does a RouterQueryEngine differ from a standard query engine?
   It can route queries to different underlying indexes based on the question's topic, enabling multi-domain RAG systems.

5. What metadata would you add to a document in a RAG system?
   Source name, date, author, category, version, or any field that helps filter results to the relevant subset.

Challenge: Build a RAG system that indexes the official documentation for three of your favorite tools. Implement metadata filtering so users can ask "Show me authentication options in [tool name]" and receive answers only from that tool's docs.

Mini Project: Document Q&A System

Build a complete RAG pipeline that indexes a set of documents and answers questions:

import os
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.storage import StorageContext
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(model="gpt-4", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

INDEX_DIR = "./my_index"

# Index or load
if os.path.exists(INDEX_DIR):
    storage_context = StorageContext.from_defaults(persist_dir=INDEX_DIR)
    index = load_index_from_storage(storage_context)
    print("Loaded existing index")
else:
    documents = SimpleDirectoryReader("./my_docs").load_data()
    index = VectorStoreIndex.from_documents(documents)
    index.storage_context.persist(persist_dir=INDEX_DIR)
    print(f"Created new index with {len(documents)} documents")

# Interactive Q&A
query_engine = index.as_query_engine(similarity_top_k=3)
print("\nAsk questions about your documents (type 'quit' to exit)\n")

while True:
    question = input("Q: ")
    if question.lower() == "quit":
        break
    response = query_engine.query(question)
    print(f"A: {response}\n")
    print(f"Sources: {[n.node.metadata.get('file_name', 'unknown') for n in response.source_nodes]}\n")

Expected output:

Created new index with 5 documents

Ask questions about your documents (type 'quit' to exit)

Q: What security features does Doda Browser have?
A: Doda Browser includes built-in tracker blocking, encrypted DNS (DoH/DoT),
    anti-fingerprinting, and automatic HTTPS upgrades.

Sources: ['doda-browser-specs.md', 'doda-security.md']

Try it: Add your own documents to ./my_docs/ and ask questions. Experiment with different chunk_size and similarity_top_k values to see how they affect answer quality.

FAQ

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.