Local RAG Over My Coursework

01 / OVERVIEW

The idea behind it.

A private question-answering system built over my entire BYU course archive: lecture PDFs, textbooks, lab writeups, Obsidian notes, essays, and source code across 33 courses. Documents are parsed, chunked, and embedded with nomic-embed-text through Ollama, then stored in a NumPy vector index. At query time the system embeds the question, retrieves the most relevant passages using hybrid dense-plus-lexical scoring, and feeds them to a local qwen2.5-coder model that answers with inline citations back to the exact source files. Deliberately built without LangChain, FAISS, or PyTorch — the runtime depends only on NumPy and Ollama, which keeps it fast, portable, and fully offline. Nothing ever leaves the machine.

02 / FEATURES

What it does.

• 01Indexes 2,157 files into 24,778 searchable chunks across 33 courses
• 02Multi-format ingestion: PDF, DOCX, Markdown, plain text, and source code
• 03Fully local inference — no API keys, no network calls, complete privacy
• 04Hybrid retrieval combining cosine similarity with lexical overlap scoring
• 05Inline citations mapping every claim back to the originating file and chunk
• 06Course-scoped and file-type-scoped filtering of the search space
• 07Incremental re-indexing that fingerprints files and skips unchanged ones
• 08Interactive chat mode with conversation memory and runtime commands
• 09Retrieval-only search mode for locating source material without invoking the LLM

03 / CHALLENGES

The problems worth solving.

• 01Extracting usable text from 358 PDFs including scanned lecture slides and full textbooks
• 02Malformed PDF math glyphs decoding to unpaired UTF-16 surrogates that crashed serialization after 20 minutes of embedding work
• 03A garbage-collection stall where building tens of thousands of records with the full corpus resident sent the cyclic collector into repeated full passes, hanging the save indefinitely at 100% CPU
• 04Duplicate files across the archive crowding out distinct sources in the retrieval results
• 05Course-specific queries retrieving poorly because note bodies rarely name their own course
• 06Building a workable stack on Python 3.14, where PyTorch and FAISS wheels were unavailable

04 / LEARNINGS

What I took away.

• 01RAG architecture end to end: ingestion, chunking, embedding, retrieval, and generation
• 02Why chunk overlap and paragraph-aware splitting matter for retrieval quality
• 03Injecting document metadata into chunk text so embeddings capture context the body omits
• 04Hybrid search — dense vectors miss exact identifiers like course codes that lexical matching catches
• 05Cosine similarity over L2-normalized vectors reduces to a single matrix multiply
• 06Profiling a hung Python process with native stack sampling to find a GC bottleneck
• 07How dataclasses.asdict deep-copies records and becomes a scaling hazard at volume
• 08Content-hash deduplication to prevent near-identical sources dominating top-k results
• 09Choosing a dependency-light design when the runtime environment constrains the options