Chat With Your Own Notes: Building an Obsidian RAG in Python

Thomas Prommer Technology Executive & AI Strategy Consultant Connect on LinkedIn

Published: June 16, 2026

Updated: June 16, 2026

You can chat with your own Obsidian notes in a couple hundred lines of Python, because the vault is already plain markdown on disk. You walk the vault for .md files, strip the frontmatter and [[wiki-links]] to clean text, split each note into overlapping word windows, embed each chunk with OpenAI's text-embedding-3-small (1536 dimensions), and store the vectors in a sqlite-vec vec0 table inside one SQLite file. To ask a question, you embed it, retrieve the nearest chunks, and hand them to gpt-4.1-mini, which answers from your notes and cites the source file. No Pinecone, no LangChain.

Why build this on an Obsidian vault at all?

Because the hardest part of most "chat with your data" projects is getting the data out of whatever app it lives in, and Obsidian doesn't have that problem. A vault is a folder of plain markdown files sitting on your disk. There is no database to dump, no API to authenticate against, no export button to babysit. You point os.walk at the directory and you are already holding your corpus.

That matters because it changes what the project is. I am not building a data pipeline; I am building a small retrieval layer over text I already own and already wrote. The content is in my own words, which is exactly what you want a personal assistant grounded in — it should sound like my thinking because it is reading my thinking. The rest of this post is the actual build, in the order the notes flow through it. I built the same stack for a published website in Build Ask Tom; here the only real difference is the corpus and the markdown cleanup it needs.

What is the full stack?

Five pieces, and only two of them cost anything to run. The source is a folder you already have, the storage is a single file, and the framework is no framework.

What is the build sequence, start to finish?

Six steps. The first four build the index from your vault; the last two are the live question. You run the first four once (and again whenever you want to re-index), and the last two on every query.

1. Walk the vault. Recurse the vault directory for .md files, keeping each note's path relative to the vault root.
2. Clean the markdown. Strip the YAML frontmatter block and flatten [[wiki-links]] to their display text, leaving readable prose.
3. Chunk. Split each note into overlapping word windows so passages near a boundary survive intact.
4. Embed & store. Embed each chunk with text-embedding-3-small and insert the vector, the text, and the note path into a sqlite-vec vec0 table.
5. Retrieve. At query time, embed the question and run a similarity search to pull the closest chunks and their note paths.
6. Generate. Hand the top chunks and the question to gpt-4.1-mini and return an answer that cites the source notes.

How do you turn a vault of markdown into clean text?

This is the only Obsidian-specific step, and it is smaller than it sounds. Walk the vault, read each file, and run two cleanups: remove the YAML frontmatter block at the top of the note (everything between the opening and closing --- fences), and flatten Obsidian's wiki-link syntax so [[Project Plan|the plan]] becomes just the plan. What is left is prose the embedder can use. You do not need a markdown AST for this — two regular expressions do it:

import os, re

def clean_markdown(text):
    # drop the YAML frontmatter block if the note starts with one
    text = re.sub(r"^---\n.*?\n---\n", "", text, count=1, flags=re.DOTALL)
    # [[link|alias]] -> alias ; [[link]] -> link
    text = re.sub(r"\[\[([^\]|]+)\|([^\]]+)\]\]", r"\2", text)
    text = re.sub(r"\[\[([^\]]+)\]\]", r"\1", text)
    return text.strip()

def read_vault(vault_dir):
    for root, _, files in os.walk(vault_dir):
        for name in files:
            if name.endswith(".md"):
                path = os.path.join(root, name)
                rel = os.path.relpath(path, vault_dir)
                with open(path, encoding="utf-8") as f:
                    yield rel, clean_markdown(f.read())

Each note comes back as a (relative_path, clean_text) pair. The path is the citation: it travels with every chunk so the answer can point you back to projects/q3-planning.md rather than to a vague "your notes."

How do you chunk and embed the notes?

The part people skip is overlap. If you cut a note into hard, non-overlapping blocks, a sentence that explains the exact thing you asked about gets split across two chunks and retrieved as neither. Overlapping windows fix that for almost no cost. The chunker is fixed-size word windows with a small slide, and each chunk keeps the note path it came from:

from openai import OpenAI
client = OpenAI()

def chunk(text, size=220, overlap=40):
    words = text.split()
    step = size - overlap
    for start in range(0, len(words), step):
        window = words[start:start + size]
        if window:
            yield " ".join(window)

def embed(texts):
    resp = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts,
    )
    return [d.embedding for d in resp.data]

# build (path, chunk_text, vector) records for the whole vault
records = []
for rel, text in read_vault(VAULT_DIR):
    chunks = list(chunk(text))
    for body, vector in zip(chunks, embed(chunks)):
        records.append((rel, body, vector))

size and overlap are the only two knobs, and for prose notes a couple hundred words with a small overlap is a sane default you rarely have to touch. In a real run you'd batch the embed calls so you aren't making one request per chunk, but the shape is this.

How does sqlite-vec store and search the vectors?

This is the piece that replaces the managed vector database. You load the extension, declare a vec0 virtual table with the embedding dimension and the columns you want to keep alongside each vector, and from then on it is just SQL. Storage is an INSERT; search is a SELECT ordered by distance:

import sqlite3, sqlite_vec
from sqlite_vec import serialize_float32

db = sqlite3.connect("vault.db")
db.enable_load_extension(True)
sqlite_vec.load(db)

# the entire "vector database": one virtual table in one file
db.execute("""
  CREATE VIRTUAL TABLE notes USING vec0(
    embedding float[1536],
    path TEXT,
    body TEXT
  )
""")

# store every chunk
for path, body, vector in records:
    db.execute(
        "INSERT INTO notes(embedding, path, body) VALUES (?, ?, ?)",
        (serialize_float32(vector), path, body),
    )
db.commit()

# query: nearest chunks to the embedded question
q_vector = embed([question])[0]
rows = db.execute("""
  SELECT path, body, distance
  FROM notes
  WHERE embedding MATCH ?
  ORDER BY distance
  LIMIT 6
""", [serialize_float32(q_vector)]).fetchall()

That LIMIT 6 is the top-k handed to the model. The whole store is the vault.db file: re-indexing the vault is deleting that file and rebuilding it, and there is no migration to run because there is no separate database.

How does it turn retrieved chunks into a cited answer?

The retrieved chunks and their note paths go into the prompt, the question goes after them, and gpt-4.1-mini answers constrained to what it was handed — and told to attribute each claim to the note it came from. Because retrieval already narrowed the context to the relevant passages, the generation model doesn't have to be large; it has to be fast and obedient:

context = "\n\n".join(f"[{path}]\n{body}" for path, body, _ in rows)

prompt = f"""Answer the question using only the notes below.
Cite the source note in parentheses after each claim, e.g. (projects/plan.md).
If the notes don't contain the answer, say so.

Notes:
{context}

Question: {question}"""

answer = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": prompt}],
).choices[0].message.content
print(answer)

The whole thing now runs as a loop: read a question from the terminal, embed it, retrieve, generate, print. You are chatting with your own notes, and every answer tells you which file it came from so you can open it and read the full context yourself. That citation is the feature — a personal-notes bot earns its keep as a retrieval shortcut, not as an oracle you take on faith.

What is the honest limitation?

It indexes the text of your notes, not the link graph. Obsidian's real power is the web of [[backlinks]] between notes — the structure that turns a pile of files into a thinking tool — and this pipeline flattens those links to plain text and throws the graph away. So it answers "what did I write about X" very well and "which notes connect to X" poorly, because that relationship lived in the links I discarded during cleanup.

The fix is a second project: index each note's outbound links as their own signal and let retrieval walk the graph, not just the prose. I deliberately left that out of the first build, because the version that reads your prose and cites the source note is the one that earns its keep on day one, and the graph layer is real work for a narrower payoff. That trade — spend complexity only where it changes the answer — is the same discipline behind every choice here: the file instead of the service, the fixed chunker instead of the clever one, the small model instead of the frontier one. For the strategy case on why a citable, searchable vault beats another note-taking app in the first place, I made the argument in Build a Second Brain That Answers Back on ctaio.dev. And if you're still deciding where your notes should live before you wire retrieval on top, the We The Flywheel Radar verdict on Obsidian vs Notion is the place to start.

FAQ