llm.nvim

English | 简体中文

Contents

• Screenshots
  • Chat
  • Quick Translation
  • Explain Code
  • Ask
  • Attach To Chat
  • Optimize Code
  • Generate Test Cases
  • AI Translation
  • Image Recognition
  • Generate Git Commit Message
  • Generate Doc String
  • Web Search
  • Diagnostic
  • Lsp
• Installation
  • Dependencies
  • Preconditions
    • Websites of different AI platforms
  • Minimal installation example
• Configuration
  • Commands
  • Model Parameters
  • keymaps
  • Tool
  • UI
  • Custom parsing function
• TODO List
• Author’s configuration
• Acknowledgments
  • Special thanks

Screenshots

Chat

models | UI

Press ? can display the shortcut key help window

• Float-UI

• Split-UI

Quick Translation

enable_cword_context = true: Translate the text under the cursor in normal mode.

Explain Code

Streaming output | Non-streaming output

Ask

One-time, no history retained.

You can configure inline_assistant to decide whether to display diffs (default: show by pressing ‘d’).

Attach To Chat

You can configure inline_assistant to decide whether to display diffs (default: show by pressing ‘d’).

Optimize Code

• Display side by side

• Display in the form of a diff

Generate Test Cases

AI Translation

Image Recognition

Generate Git Commit Message

Generate Doc String

Web Search

/buffer | /file | @web_search

Diagnostic

Both disposable_ask_handler, attach_to_chat_handler, side_by_side_handler and action_handler can enable diagnostic features:

diagnostic = { min = vim.diagnostic.severity.HINT },
-- or
-- diagnostic = { vim.diagnostic.severity.WARN, vim.diagnostic.severity.ERROR },
-- see `:h diagnostic-severity`

Lsp

Both disposable_ask_handler, attach_to_chat_handler, and action_handler can enable lsp features:

lsp = {
  cpp = { methods = { "definition", "declaration" } },
  python = { methods = { "definition" } },
  lua = { methods = { "definition", "declaration" } },

  root_dir = { {'pyproject.toml', 'setup.py' }, ".git" },
},

⬆ back to top

Installation

Dependencies

• curl
• fzf >= 0.37.0: Optional. Split style preview of session history and image recognition tool image selection depends on fzf(The author’s development environment is 0.39.0)
• render-markdown.nvim: Optional. Better Markdown preview depends on this plugin.

{
    "MeanderingProgrammer/render-markdown.nvim",
    dependencies = {
      {
        "nvim-treesitter/nvim-treesitter",
        branch = "main",
        config = function()
          vim.api.nvim_create_autocmd("FileType", {
            pattern = { "llm", "markdown" },
            callback = function()
              vim.treesitter.start(0, "markdown")
            end,
          })
        end,
      },
      "nvim-mini/mini.icons",
    }, -- if you use standalone mini plugins
    ft = { "markdown", "llm" },

    config = function()
      require("render-markdown").setup({
        restart_highlighter = true,
        heading = {
          enabled = true,
          sign = false,
          position = "overlay", -- inline | overlay
          icons = { "󰎤 ", "󰎧 ", "󰎪 ", "󰎭 ", "󰎱 ", "󰎳 " },
          signs = { "󰫎 " },
          width = "block",
          left_margin = 0,
          left_pad = 0,
          right_pad = 0,
          min_width = 0,
          border = false,
          border_virtual = false,
          border_prefix = false,
          above = "▄",
          below = "▀",
          backgrounds = {},
          foregrounds = {
            "RenderMarkdownH1",
            "RenderMarkdownH2",
            "RenderMarkdownH3",
            "RenderMarkdownH4",
            "RenderMarkdownH5",
            "RenderMarkdownH6",
          },
        },
        dash = {
          enabled = true,
          icon = "─",
          width = 0.5,
          left_margin = 0.5,
          highlight = "RenderMarkdownDash",
        },
        code = { style = "normal" },
      })
    end,
  }

Preconditions

1. Register on the official website and obtain your API Key (Cloudflare needs to obtain an additional account).

2. Set the LLM_KEY (Cloudflare needs to set an additional ACCOUNT) environment variable in your zshrc or bashrc.

export LLM_KEY=<Your API_KEY>
export ACCOUNT=<Your ACCOUNT> # just for cloudflare

Websites of different AI platforms

For local llms, Set LLM_KEY to NONE in your zshrc or bashrc.

⬆ back to top

Minimal installation example

• lazy.nvim

  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim"},
    cmd = { "LLMSessionToggle", "LLMSelectedTextHandler", "LLMAppHandler" },
    config = function()
      require("llm").setup({
        url = "https://models.inference.ai.azure.com/chat/completions",
        model = "gpt-4o-mini",
        api_type = "openai"
      })
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
    },
  }

• Mini.deps

require("mini.deps").setup()
MiniDeps.add({
        source = "Kurama622/llm.nvim",
        depends = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
        cmd = { "LLMSessionToggle", "LLMSelectedTextHandler", "LLMAppHandler" },
})

require("llm").setup({
        url = "https://models.inference.ai.azure.com/chat/completions",
        model = "gpt-4o-mini",
        api_type = "openai"
})

Configure template

Configuration

Commands

Model Parameters

keymaps

Tool

Each handler’s parameters can be referred to here.

Examples can be seen AI Tools Configuration

UI

See UI Configuration and nui/popup

⬆ back to top

Custom parsing function

For streaming output, we use our custom streaming_handler; for AI tools that return output results in one go, we use our custom parse_handler.

Below is an example of ollama running llama3.2:1b.

local function local_llm_streaming_handler(chunk, ctx, F)
  if not chunk then
    return ctx.assistant_output
  end
  local tail = chunk:sub(-1, -1)
  if tail:sub(1, 1) ~= "}" then
    ctx.line = ctx.line .. chunk
  else
    ctx.line = ctx.line .. chunk
    local status, data = pcall(vim.json.decode, ctx.line)
    if not status or not data.message.content then
      return ctx.assistant_output
    end
    ctx.assistant_output = ctx.assistant_output .. data.message.content
    F.WriteContent(ctx.bufnr, ctx.winid, data.message.content)
    ctx.line = ""
  end
  return ctx.assistant_output
end

local function local_llm_parse_handler(chunk)
  local assistant_output = chunk.message.content
  return assistant_output
end

return {
  {
    "Kurama622/llm.nvim",
    dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
    cmd = { "LLMSessionToggle", "LLMSelectedTextHandler" },
    config = function()
      require("llm").setup({
        url = "http://localhost:11434/api/chat", -- your url
        model = "llama3.2:1b",

        streaming_handler = local_llm_streaming_handler,
        app_handler = {
          WordTranslate = {
            handler = tools.flexi_handler,
            prompt = "Translate the following text to Chinese, please only return the translation",
            opts = {
              parse_handler = local_llm_parse_handler,
              exit_on_move = true,
              enter_flexible_window = false,
            },
          },
        }
      })
    end,
    keys = {
      { "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
    },
  }
}

⬆ back to top

TODO List

todo-list

⬆ back to top

Author’s configuration

plugins/llm

Acknowledgments

We would like to express our heartfelt gratitude to the contributors of the following open-source projects, whose code has provided invaluable inspiration and reference for the development of llm.nvim:

• olimorris/codecompanion.nvim: Diff style and prompt.
• SmiteshP/nvim-navbuddy: UI.
• milanglacier/minuet-ai.nvim: Code completions.

Special thanks

ACKNOWLEDGMENTS