LLM powered development for Neovim
Important
This is currently a work in progress, expect things to be broken!
llm.nvim is a plugin for all things LLM.
This project is influenced by copilot.vim and tabnine-nvim
Formerly hfcc.nvim.
Features
Code completion
This plugin supports "ghost-text" code completion, à la Copilot.
Choose your model
Requests for code generation are made via curl
to an HTTP endpoint.
You can use the Hugging Face Inference API or your own HTTP endpoint, provided it adheres to the API specified here.
Note
Requests via curl
are a legacy feature. This plugin is evolving to become a front-end for llm-ls
.
Install
With Inference API
-
Create and get your API token from here https://huggingface.co/settings/tokens.
-
Define how the plugin will read your token. For this you have multiple options, in order of precedence:
- Pass
api_token = <your token>
in plugin opts - this is not recommended if you use a versioning tool for your configuration files - Set the
LLM_NVIM_API_TOKEN
environment variable - You can define your
HF_HOME
environment variable and create a file containing your token at$HF_HOME/token
- Install the huggingface-cli and run
huggingface-cli login
- this will prompt you to enter your token and set it at the right path
- Pass
-
Choose your model on the Hugging Face Hub, and, in order of precedence, you can either:
- Set the
LLM_NVIM_MODEL
environment variable - Pass
model = <model identifier>
in plugin opts
- Set the
With your own HTTP endpoint
All of the above still applies, but note:
-
When
api_token
is set, it will be passed as a header:Authorization: Bearer <api_token>
. -
Instead of setting a Hugging Face model identifier in
model
, set the URL for your HTTP endpoint.
packer
Usingrequire("packer").startup(function(use)
use {
'huggingface/llm.nvim',
config = function()
require('llm').setup({
-- cf Setup
})
end
}
end)
lazy.nvim
Usingrequire("lazy").setup({
{
'huggingface/llm.nvim',
opts = {
-- cf Setup
}
},
})
vim-plug
UsingPlug 'huggingface/llm.nvim'
require('llm').setup({
-- cf Setup
})
Setup
local llm = require('llm')
llm.setup({
api_token = nil, -- cf Install paragraph
model = "bigcode/starcoder", -- can be a model ID or an http(s) endpoint
-- parameters that are added to the request body
query_params = {
max_new_tokens = 60,
temperature = 0.2,
top_p = 0.95,
stop_token = "<|endoftext|>",
},
-- set this if the model supports fill in the middle
fim = {
enabled = true,
prefix = "<fim_prefix>",
middle = "<fim_middle>",
suffix = "<fim_suffix>",
},
debounce_ms = 150,
accept_keymap = "<Tab>",
dismiss_keymap = "<S-Tab>",
max_context_after = 5000,
max_context_before = 5000,
tls_skip_verify_insecure = false,
-- llm-ls integration
lsp = {
enabled = false,
bin_path = vim.api.nvim_call_function("stdpath", { "data" }) .. "/llm_nvim/bin/llm-ls",
},
tokenizer_path = nil, -- when setting model as a URL, set this var
context_window = 8192, -- max number of tokens for the context window
})
Commands
LLMToggleAutoSuggest
which enables/disables ghost text completion