Obsidian-copilot

How would a copilot for writing and thinking look like?

Here's a possible implementation: You write a section header and the copilot retrieves relevant notes & docs to draft that section for you. This pattern of retrieval-augmented generation can also be extended to other use cases. Here's an example where the copilot helps you reflect on your week based on your daily journal entries.

Currently, copilot helps you:

• Draft sections based on your notes
• Reflect on your week based on your daily journal entries

More technical details on how it works here: Obsidian-Copilot: A Prototype Assistant for Writing & Thinking

Quick start

Clone and update the path to your obsidian-vault and huggingface hub cache

git clone https://github.com/eugeneyan/obsidian-copilot.git

# Open Makefile and update the following paths
export OBSIDIAN_PATH = /Users/eugene/obsidian-vault/
export TRANSFORMER_CACHE = /Users/eugene/.cache/huggingface/hub

Build the OpenSearch and semantic indices

# Build the docker image
make build

# Start the opensearch container and wait for it to start. 
# You should see something like this: [c6587bf83572] Node 'c6587bf83572' initialized
make opensearch

# In ANOTHER terminal, build your artifacts (this can take a while)
make build-artifacts

Running the retrieval app

# First, stop the opensearch container (CTRL + C). Then, start the retrieval app. 
# You should see this: Uvicorn running on http://0.0.0.0:8000
make run

Install the copilot-plugin, enable it in community plugin settings, and update the API key in copilot

make install-plugin

How does it work?

At a high level, when you type a section header, it'll:

• Retrieve relevant documents/snippets from the your obsidian vault via:
  • Keyword search (opensearch)
  • Semantic search (semantic retrieval)
• The retrieved context is then used to generate paragraphs for the section
• It is also displayed in a new tab for info

TODOs

• Add support for using anthrophic claude (100k context)
• Assess sending in entire documents instead of chunks