The whole local AI stack in one executable: it runs and manages the models, and searches everything you own with them. Project site · Tutorial reels · PyPI · Obsidian plugin · REST API · Chat (#lilbee) lilbee runs and manages your models: chat, embedding, vision, and rerank, placed across every GPU you have. It puts them to work as a search…
The whole local AI stack in one executable: it runs and manages local AI models across every GPU, and it's a search engine you can talk to, with cited answers from your files, code, and the web. MCP server for coding agents, web crawler, TUI, CLI, REST API, Python library. No Ollama or LM Studio needed, works with both.
lilbee runs and manages your models: chat, embedding, vision, and rerank, placed across every GPU you have. It puts them to work as a search engine you can talk to, over your files, notes, code, and the web, where every answer cites the exact file and line. It crawls websites into your library, launches your coding agents on local models, and hands any MCP-aware agent cited answers from everything you’ve indexed. The same engine backs the Obsidian community plugin, so your vault gets all of it without a terminal. Ask in plain English. No containers, no networking, nothing else to install or set up.
And it is private. Your files, the index, the embeddings, your questions, and the answers stay on your machine. lilbee sends no telemetry and needs no account. It contacts a cloud provider only when you set an API key for it, in lilbee’s settings or in the provider’s own environment variable. Then it checks once that the key works, and sends your questions only to a cloud model you pick.
It’s all one program: no separate model server, vector database, or container to stand up. lilbee runs the models and keeps the index itself. Reach it as a terminal app, CLI, Model Context Protocol server, HTTP API, or Python library. Close it and it’s gone, or run it as a service to keep it warm. Everything runs on your computer; it uses a cloud model only when you pick one.
Models are no different: lilbee has its own model manager and multi-GPU fleet, built on llama.cpp, so one executable does everything (browse Hugging Face, download a model, give it a role, run it on Metal / Vulkan / CUDA). You don’t need Ollama or LM Studio at all: the architectures lilbee runs are behind most of the 190,000+ GGUF repos on Hugging Face, with representatives verified on real GPUs. If you already use them, point lilbee at your existing setup and keep your models.
Tutorial reel: every demo on this page, plus the extras that don’t fit here, with long-form captions at lilbee.sh/tutorial.
⚠️ Beta software
lilbee is in active beta development. Every release on PyPI is a pre-release; you must use --pre (or uv’s --prerelease=allow) when installing. Interfaces, command names, and on-disk formats may shift between betas. Feedback, bug reports, and issues are very welcome; that’s the whole point of the beta.
A self-contained bundle (start here): the standalone binary, or the Homebrew / AUR / Nix / Docker / Flatpak / Snap / Scoop builds that wrap it. It carries its own Python runtime, llama.cpp, the model engine, and the optional extras, so there is nothing else to install and nothing to assemble. The trade-off is one large download and a small cold-start cost the first time it self-extracts.
Into your own Python with pip or uv (Python 3.11 to 3.14), in its own section below. This is the developer route: it installs lilbee as a library in an environment you manage, and the model engine is a separate extra from a separate index that you have to ask for. Take it if you are working on lilbee or importing it into your own code.
Have a discrete GPU? The bundled builds and the Vulkan engine already use it. There are faster vendor builds too: NVIDIA (CUDA) and AMD (ROCm).
No external services either way; lilbee downloads and runs models locally. Optional, for scanned-PDF / image OCR: Tesseract (brew install tesseract / apt install tesseract-ocr) or a GGUF vision model.
How
Command
Notes
Homebrew
brew tap tobocop2/lilbee && brew install lilbee
macOS (Apple silicon and Intel) / Linux x86_64. Bundled build; clears the macOS quarantine flag for you.
AUR
paru -S lilbee
Arch Linux. Wraps the Linux x86_64 binary; works with yay / pacaur / any helper.
Docker
docker run --rm -v lilbee-data:/home/lilbee/data ghcr.io/tobocop2/lilbee:latest --help
GHCR image, tagged by version and latest. Data lives at /home/lilbee/data. Mount a volume there. :cuda and :rocm tags carry the vendor builds.
Nix
nix run github:tobocop2/lilbee
NixOS, nix-darwin, or any host with nix. On Linux the flake bundles glibc, libgomp, and vulkan-loader so it runs on bare NixOS.
Linux x86_64, any distro with flatpak. Needs the Flathub remote for the runtime. Run with flatpak run io.github.tobocop2.lilbee, or make it a plain command: alias lilbee='flatpak run io.github.tobocop2.lilbee' in your shell rc (swap in .cuda / .rocm / .compat for a vendor build). flatpak update picks up new releases. Data lives under ~/.var/app/io.github.tobocop2.lilbee/. Prefer one file? Grab lilbee.flatpakref from the release and flatpak install ./lilbee.flatpakref.
Windows x86_64. Scoop needs git to add a bucket, so the first command installs it (a no-op when present). Reads your NVIDIA driver version and installs the matching CUDA build (555.85+ gets cu125, 551.61+ gets cu124), otherwise the Vulkan build. It says which one it picked. scoop update lilbee upgrades. Added the bucket before 2026-09-08? It moved to its own repository, so run scoop bucket rm lilbee and add it again; your install is untouched.
npm
npm install -g lilbee
macOS / Linux x86_64 / Windows, Node 18+. A small launcher: on first run it detects your hardware (CUDA, ROCm, Metal, or the AVX-baseline build), downloads the matching standalone binary of the latest release, verifies it, and caches it. lilbee prepare does that ahead of time; npx -y lilbee mcp runs the MCP server for agent hosts.
One file, own Python runtime, no pip needed. Linux needs glibc 2.28+; the macOS / Windows builds are unsigned (xattr -d com.apple.quarantine ./lilbee-macos-arm64 if Gatekeeper blocks it).
On NVIDIA hardware
The default Vulkan build works on NVIDIA cards, but there’s a dedicated CUDA build that’s faster on NVIDIA hardware and sidesteps the iGPU + dGPU Vulkan-loader crash on Windows.
Same lilbee command after install. The CUDA runtime is bundled; you only need the NVIDIA driver. Already have the regular lilbee installed? On AUR paru -S lilbee-cuda swaps it automatically; on Homebrew run brew uninstall lilbee first, and on Scoop run scoop uninstall lilbee first. Older driver? cu124 and cu121 ship via the matching wheel indexes and as direct-download Linux binaries on the release page.
On AMD hardware
The default Vulkan build works on AMD cards, and stays the fallback if ROCm isn’t set up. There’s also a dedicated ROCm build that’s faster. It bundles the ROCm userspace, so you need only the amdgpu kernel driver.
Same lilbee command after install. Linux only. Cards: MI100, MI200, MI300, MI350, RDNA2, RDNA3, RDNA3.5 APUs and RDNA4. ROCm 7 ships no GEMM kernels for the MI50, so it needs the Vulkan build. Largest of the three builds, since the ROCm userspace and per-card kernels ship inside it.
Then check it runs and pick a model:
lilbee self-check # ~90 MB download; runs an inference + an embedding; "SELF-CHECK PASSED" on success
lilbee # launch the terminal app; pick your models from the catalog on first run
The usage guide covers the rest: TUI screens, slash commands, CLI, HTTP server, MCP, env vars, and config.toml. The settings reference lists every setting and which of those surfaces can change it.
On older CPUs (pre-AVX2)
Pre-2013 Intel or pre-Zen AMD CPUs lack AVX2, so the normal build crashes on launch. The lilbee-compat build runs on any x86-64 chip back to ~2008.
Install commands for every channel, and why it's needed
pip install --pre lilbee 'lancedb==0.40.0+compat' --extra-index-url https://lilbee.sh/compat/ — no engine: there is no compat engine wheel, so use a bundled build above or bring your own llama-server
Use a bundled build here if you can. The compat index carries the patched lancedb only, so the pip row installs lilbee without an engine and every model call will fail until you point LILBEE_LLAMA_SERVER_PATH at a llama-server that runs on your CPU.
An older CPU with an NVIDIA GPU (CPU with AVX, 2011+): lilbee-compat-cuda is the compat build with CUDA 12.4, so it needs NVIDIA driver 550+ on Linux and 551.61+ on Windows. The CUDA runtime is bundled.
Same lilbee command after install. The crash is from lancedb’s AVX2-compiled wheels; this build swaps in a lancedb fork that picks instructions at runtime. A 👍 or comment on the upstream lance PR helps it land.
Linux runtime requirements
The Linux x86_64 wheel and binary bundle the Vulkan loader, so they start on any distro. GPU detection still reads adapters through the system loader, though: without libvulkan1 lilbee treats the machine as CPU-only. Most desktop distros (Ubuntu 22.04+, Pop!_OS, Mint) ship it; on bare Arch / Fedora / Alpine images install it once: sudo pacman -S vulkan-icd-loader (Arch / Manjaro), sudo dnf install vulkan-loader (Fedora, RHEL), or sudo apt-get install libvulkan1 (Debian, Ubuntu).
Optional extras
These only matter for a pip or uv install: add the name in brackets, e.g. pip install --pre 'lilbee[engine,crawler,providers]' (combine multiple, and --extra-index-url still works). The standalone binary and the Homebrew / AUR / Nix / Docker / Flatpak / Snap builds already include all four, which is why they need none of this. [engine] is the one that is not really optional; lilbee works fine without the other three.
Extra
What it adds
[engine]
The bundled llama-server that runs your models. Published on lilbee.sh rather than PyPI, so it needs the --extra-index-url for your hardware; without it nothing can load a model.
[crawler]
Index websites alongside your files: crawl a docs site or wiki to markdown, then search it offline.
[providers]
Bridge to hosted model providers for chat, vision, or embeddings while other roles stay local. The TUI flags when a hosted role is active.
[graph]
Concept-graph search: extracts the ideas in your documents and uses how they relate to surface matches plain keyword search misses. No extra model calls.
For working on lilbee, or importing it as a library into an environment you
manage. Everything below needs the [engine] extra and its index; the bundled
builds above do not.
pip / uv install commands, and the engine index per hardware
Same wheels as pip; fetches a Python for you if you need one.
From source
git clone https://github.com/tobocop2/lilbee && cd lilbee && uv sync && uv run lilbee
Needs git and uv. uv sync resolves the in-repo engine package, but CI is what fills its bin/, so a checkout has no engine binaries: install the [engine] extra from the index below, or point LILBEE_LLAMA_SERVER_PATH at your own llama-server.
The engine.llama-server runs every model, and it ships as the [engine]
extra rather than as part of lilbee. It is not on PyPI: the CUDA and ROCm
wheels run 444 MiB to 863 MiB each, several times
PyPI’s default 100 MiB per-file limit,
so every backend is published from lilbee’s own
PEP 503 package index at lilbee.sh. That
is what --extra-index-url is for, and the index you pick is the build you get:
cpu is compiled with every GPU backend off, metal publishes only a macOS
arm64 wheel, vulkan only Linux and Windows ones.
Leave the extra off and lilbee still installs and starts. The first thing that
needs a model then fails with an error naming the engine and repeating the
command for your hardware, so it is recoverable rather than mysterious, but it
is a step you have to take.
Hardware requirements
Standalone mode needs nothing but your machine. No cloud required. Minimum: Apple Silicon Mac, or a 64-bit Intel/AMD CPU from 2013+ (older CPUs: On older CPUs), or an ARMv8 Linux box; 8 GB RAM, 2 GB disk.
Full platform and resource breakdown
Platform
Minimum
Recommended
macOS arm64
Apple Silicon (M1 or newer), macOS 11+
M-series Pro / Max / Ultra
macOS x86_64
Intel Mac with AVX (Ivy Bridge or newer), macOS 12+
Modern Intel/AMD CPU + an NVIDIA, AMD, or Intel Arc GPU
Windows x86_64
64-bit Intel/AMD from 2013+ (x86-64-v3), Windows 10/11
Modern desktop / workstation CPU + GPU
Linux ARM64
ARMv8 NEON-capable (Raspberry Pi 4+, AWS Graviton, Ampere Altra)
Modern ARM server with 16+ GB RAM
Resource
Minimum
Recommended
RAM
8 GB
16 to 32 GB to keep several local models warm at once (chat + embed + rerank + vision); actual footprint scales with the sizes and quantizations you pick
GPU / Accelerator
none required (CPU-only works)
Apple Silicon (Metal) · NVIDIA / AMD / Intel Arc (Vulkan) · faster vendor builds with the runtime bundled: CUDA on NVIDIA, ROCm on AMD (see Install)
Disk
2 GB
10+ GB for multiple models
Quick start
Two recommended ways to use lilbee, depending on whether you’re the one driving:
Run lilbee for the full-screen terminal app. On first run you pick a chat model straight from the catalog, fit-checked against your hardware, then you index files, search, and chat without leaving the TUI. The Settings screen exposes every retrieval knob (search depth, distance threshold, reranker, chunking) so you can tune lilbee to your library shape.
Connect it to your agent over MCP. Any MCP-aware coding agent calls lilbee_search / lilbee_add and gets back cited snippets it can quote. Agents can also fine-tune lilbee on the fly via lilbee_settings_set. Drop in the lilbee-mcp skill and the agent reads the full surface: every tool, every retrieval knob, and when to widen for prose vs narrow for code. See A reference for AI agents.
Retrieval defaults are sane, and every setting is tunable from env vars or config.toml, most of them from the TUI, MCP, and the HTTP API too; the settings reference has a column per surface. The CLI and HTTP API cover scripting and headless runs. See the usage guide.
Highlights
Private by default. The models, the index, and every question and answer stay on your machine. No telemetry, no account. lilbee contacts a cloud provider only when you set its API key: once to check the key, then only for a cloud model you pick.
Answers cite the source line. Click a citation, jump to the file at the exact line; when the answer isn’t in your library, lilbee says so instead of inventing one.
It works, and the demos prove it. Every GIF and reel here is recorded live on real hardware, nothing staged, backed by 100% test coverage, full typing, and CI on macOS, Linux, and Windows.
One command to running. Install, run lilbee, pick a model that fits your machine from the catalog, and you’re chatting.
Reads almost anything:90+ formats and 150+ languages across documents, scanned pages, spreadsheets, ebooks, web pages, and source code.
Chunks that stand on their own.Prose and code are split differently so each piece keeps its meaning, which is where most of the retrieval quality lives.
A real search engine on top, ranking every result by how well it answers you, with 50+ tunable knobs and sane defaults.
It brings and runs the models itself, on Metal, Vulkan, or CUDA, with no server to point at and no cloud account. Browse Hugging Face, pull a model, give it a role (chat, embedding, vision, rerank).
A model too big for one card runs across all of them, sized with gguf-parser and tensor-split automatically, or pinned by hand. Run a model bigger than one card.
Already on Ollama or LM Studio? Keep them. lilbee’s own manager covers the same architectures they run, and their models also show up in the same pickers.
One install, many surfaces: TUI, CLI, MCP server, REST API, and Python library, so your coding agent answers from your real files, with citations.
Everything in one file, nothing to operate. The binary bundles the whole stack (search engine, crawler, MCP + HTTP servers, TUI, Python, llama.cpp) in ~360-530 MB, or ~0.7-1.3 GB with CUDA; it loads on demand and nothing stays running.
Per-project libraries. One library for everything, or one per project.
Why lilbee
A small local model is fun, but limited on its own. Give it properly processed documents and a search engine over them, and it becomes genuinely powerful. Without those, it never gets past novelty.
lilbee does all of it in one install: it runs the models, processes your documents, crawls the web pages you point it at, and searches the lot with a real engine. The same engine works two ways:
An Encarta 99 over your own files. Build a library from your documents and saved web pages, then read it and ask questions of it in the terminal.
A reference layer for code. Point it at your project, dependencies, and API docs, and your coding agent answers from what’s actually there, with file:line citations, instead of guessing function names.
The long-term goal: make local AI genuinely useful on hardware you already own, with no token budgets, no provider to depend on, and nothing about your files leaving your machine; the cloud’s there only when you want it.
How lilbee compares
lilbee is built for consumer hardware and for people who don’t want to babysit infrastructure. It isn’t another model server you point an app at; it’s a local search engine with the model runner built in. One install gives you the whole stack in a single executable:
A search engine over your files, with answers that cite the source line, not just a model to chat with.
A managed fleet, chat, embedding, vision, and reranking, spread across every GPU in the machine behind a load-balancing router, not one model loaded at a time.
Everything bundled: model manager, search engine, web crawler, MCP server for coding agents (native opencode and hermes), HTTP server, TUI, and Python, in one file.
It sits between two worlds: the desktop runners that get a model chatting on your machine (Ollama, LM Studio), and vLLM, the server you stand up to push one model to a cluster of users. lilbee runs models to do retrieval over your files, and places that whole stack across the GPUs in the machine.
Full comparison table: lilbee vs Ollama, LM Studio, and vLLM. Click to expand.
Of the four, lilbee is the only one built around retrieval. It places chat, embedding, vision, and reranking together across the GPUs in the machine, and spreads indexing across every card.
Install size by platform: one file that undercuts the others while doing more. Click to expand.
Download sizes in decimal GB/MB (bytes ÷ 1000), measured from each project’s own release artifacts, linked. Measured 2026-11-11 from each project’s latest release: lilbee v0.6.90b692, Ollama v0.40.16, LM Studio 0.4.28-2. The files are lilbee’s macos-arm64, windows-x86_64.exe and linux-x86_64 executables (CUDA: cu125), Ollama’s Ollama.dmg, OllamaSetup.exe and ollama-linux-amd64.tar.zst, and LM Studio’s arm64 .dmg, x64 .exe and x64 .AppImage.
Even lilbee’s CUDA build stays under Ollama’s, and it’s the whole stack, not just a model runner.
Already on Ollama or LM Studio? lilbee runs on top of them. Prefer a GUI to the terminal? The Obsidian plugin maps lilbee’s model manager and search to a visual interface inside your vault.
First start
The very first launch does one-time work: a bundled build unpacks itself behind
a progress bar, and the model loads before your first answer, shown live inside
the answer bubble. Every launch after that opens straight to chat in a couple
of seconds.
Very first launch
Every launch after
Measured with a small chat model (Qwen3 0.6B):
Very first launch
Every launch after
Chat on screen
15 to 20s, one-time unpack
2 to 3s
First answer of the session
10 to 20s, the engine load plays in the bubble
the same, or instant with Keep engine warm inside the idle window
Answers after that
model speed
model speed
Bigger models load longer; the bubble shows real progress while the weights load.
The usage guide explains the engine
lifecycle and how to keep the engine warm so relaunches skip the load.
What you can do with it
A library of your own files
Point lilbee at a folder of PDFs, notes, ebooks, or code and it builds a searchable library, with citations that click back to the source line. The pattern works for anything you have a lot of text about: a shelf of appliance manuals, a field’s research papers, a car’s service manuals, your company’s internal wiki. Whatever you give it becomes searchable, and you can talk to it.
Ask it something with a real answer at stake and you get the answer, not a paraphrase of the question. Here it reads the towing section out of a car manual and answers with the page it came from, so you can check it. The panel on the right is the GPU doing it, on this machine.
Install the [crawler] extra, point lilbee at a docs site, a wiki, or a vendor’s API reference, and the pages get fetched, converted to markdown, and added to your library. From then on you can search or chat with that copy of the site offline, even after it changes or goes down.
Or crawl a whole site, not just one page. With recursive crawling on, lilbee follows the links and indexes the lot; watch the page count climb in the Task Center, then ask one question that synthesizes across the whole site.
Prose and structured documents (PDFs, Office files, ebooks, HTML, 90+ formats) go through [Xberg] with heading-aware chunking, so each chunk keeps its section context.
Code goes through [tree-sitter]'s AST-aware splitter across 150+ languages, so chunks map to functions, classes, and modules instead of arbitrary line ranges.
Scanned PDFs and photos go through OCR in 100+ languages: Tesseract for plain text (set LILBEE_OCR_LANGUAGE, e.g. eng+deu), or a local / remote vision model that keeps tables and layout as markdown.
Retrieval returns things that make sense on their own, not fragments cut through an argument or a function signature.
Pick and tune your models
You install and switch chat, embedding, vision, and reranking models from inside the terminal: browse the catalog, pull a model, pick a role. Retrieval and generation expose 50+ settings (chunk size, search strictness, reranker depth, and more), editable from the TUI, env vars, or a project-local config file. Sane defaults.
Tested GPUs and backends
Placement reads what the engine reports about your hardware, and every backend words that differently, so lilbee checks each backend on real silicon rather than inferring it from the last one. CUDA from an RTX 3090 up to H200, and up to eight A100s at once, Vulkan on NVIDIA and Intel, ROCm on an AMD Instinct MI300X, Metal on Apple Silicon, and a CPU-only host. docs/tested-gpus.md lists every machine and what each run settled. Send captures from hardware the table does not list.
Supported models
lilbee’s engine is llama.cpp, so lilbee runs what llama.cpp runs: any GGUF model built on one of the architectures below. The list comes from the bundled engine itself and grows with every engine update.
All 168 supported model architectures. Click to expand.
afmoe
apertus
arcee
arctic
arwkv7
baichuan
bailingmoe
bailingmoe2
bailingmoe3
beam
bert
bitnet
bloom
chameleon
chatglm
clef
clip
clover_moe
codeshell
cogvlm
cohere2
cohere2moe
command-r
dbrx
deci
deepseek
deepseek2
deepseek2-ocr
deepseek32
deepseek4
deepseek41
dflash
dots1
dots3note
dream
eagle3
ernie4_5
ernie4_5-moe
eurobert
exaone
exaone-moe
exaone4
falcon
falcon-h1
flint
gemma
gemma-embedding
gemma-embedding2
gemma2
gemma3
gemma3n
gemma4
gemma4-assistant
glm-dsa
glm4
glm4moe
glm5-next
gpt-oss
gpt2
gptj
gptneox
granite
granite_swa
granitehybrid
granitemoe
graniteswitch
grok
grovemoe
hrm_text
hunyuan-dense
hunyuan-moe
hunyuan_vl
hy_v3
hy_v4
internlm2
jais
jais2
jamba
jina-bert-v2
jina-bert-v3
k2-horizon
kimi-k3
kimi-linear
kolibri1
laguna
lfm2
lfm2moe
lfm3
limite
llada
llada-moe
llama
llama-embed
llama4
maincoder
maion-coder
mamba
mamba2
maple
mellum
mimo2
minicpm
minicpm3
minimax-01
minimax-m2
minimax-m3
mistral-large4
mistral3
mistral4
modern-bert
mpt
muse-glimmer
muse-spark
nanbeige
nemotron
nemotron_h
nemotron_h_moe
neo-bert
nomic-bert
nomic-bert-moe
olmo
olmo2
olmoe
openelm
orion
paddleocr
pangu-embedded
phi2
phi3
phimoe
plamo
plamo2
plamo3
plm
pockettts
qwen
qwen2
qwen2moe
qwen2vl
qwen3
qwen35
qwen35moe
qwen3moe
qwen3next
qwen3tts
qwen3vl
qwen3vlmoe
qwen4exp
refact
rnd1
rwkv6
rwkv6qwen2
rwkv7
seed_oss
sensenova_u1
smallthinker
smollm3
spark2_5
stablelm
starcoder
starcoder2
step35
t5
t5encoder
talkie
vllm-decision2
wavtokenizer-dec
xverse
One representative per family also runs the full pipeline end to end on real GPUs – the models, method, and results are in docs/tested-models.md.
lilbee works with Ollama and LM Studio. Finding and running models for you is the default and simplest path (lilbee pulls them and runs them on Metal / Vulkan / CUDA, no server to stand up), but you don’t have to adopt a new model manager to use lilbee.
Point it at a running manager. Your models in Ollama or LM Studio show up in the same catalog and role pickers (chat, embedding, vision, rerank), labeled by where they run, alongside lilbee’s own and any cloud models. Mix freely.
They stay read-only. lilbee lists and runs them but never pulls or deletes them, so their lifecycle stays in the app you already use.
On pip / uv, this needs the [providers] extra (pip install --pre 'lilbee[providers]'); the Homebrew, AUR, Nix, Docker, Flatpak, and Snap builds already include it. See Install.
Both reels run chat and embedding on the other manager’s models, picked out of the catalog’s Library tab where each row names the server it runs on.
See when a model won’t load before you download it
Hugging Face has thousands of GGUFs, but the bundled llama.cpp only supports a subset of architectures and brand-new ones take time to reach the pinned runtime. lilbee tags incompatible models in the catalog and refuses the download (with an override confirm), so you don’t wait through a multi-GB pull only to hit “unsupported architecture” at load.
lilbee runs entirely on your machine until you set a cloud provider’s API key. Two ways to use a cloud model when you want one:
Bring your own key. Install the [providers] extra, add an API key, then point any role (chat, embedding, vision, rerank) at a cloud model from the same catalog. The TUI shows a warning the whole time a cloud model is on.
Pair lilbee with a cloud agent over MCP. Your files, the embeddings, and the index stay local. Any MCP-aware agent calls lilbee_search / lilbee_add and gets back cited snippets.
Either way, your files and the index stay on your computer. Only what you ask and the snippets needed to answer it get sent to the cloud model.
Agents
lilbee serves your coding agents two ways, both local: it runs the models they talk to, and it answers their searches with cited snippets from your library.
Launch your coding agent on local models
lilbee launch opencode, lilbee launch hermes, and lilbee launch claude set up lilbee’s local models in your agent in one command. For opencode and hermes, lilbee registers itself as a provider and an MCP server in the agent’s own config and leaves your existing setup intact; for Claude Code, lilbee serves an Anthropic-compatible API and points the session at it by env, without touching Claude Code’s settings. Each launch warms a model and opens the agent pointed at it. No API keys, no provider setup, and nothing leaves your machine, except a check of any provider API key you have set. Tool-calling works across many GGUF families; docs/agent-models.md has the verified list and how the QA harness measures it.
One model serves as many agents as you want to run. These reels show four working at once against a single local model, each in its own worktree, reading and searching this repository through lilbee.
Gemma 4 26B is a thinking model, and lilbee streams its reasoning as visible text: every pane shows the model working through the problem before it answers.
And the same setup scales up: Devstral 2 123B, 70GB across two A100 80GB cards, one agent at the model’s full 14 tokens a second.
It tunes itself, too. Tell a small local model to widen lilbee’s search when a first result comes back thin, and the second pass returns full function bodies with file:line citations; a more capable model does the same from a prompt like “improve your search results.” The lilbee-mcp skill teaches your own model the pattern.
Once configured, lilbee plugs into whatever agent you use, over MCP. Feed it your project’s docs, your dependency source, your API docs, your design notes; the agent stops making up function names and instead reads the actual code, cites file and line, and says it doesn’t know when the answer isn’t in your library. The lilbee-mcp skill is the single entry point: drop it into .opencode/skills/ or .claude/skills/ and it documents every tool, the workflows the agent should follow, and points to drop-in AGENTS.md and worker-subagent starters under examples/agent-integration/. This works with cloud-model agents too; lilbee stays local, and only the queries and the returned chunks reach the model.
Your files, the search index, and the embeddings stay on your computer. The agent calls lilbee_search and gets back cited snippets with file and line. The agent reels above show this live: each pane reads and searches this repository through lilbee.
Run a model bigger than one card
When a chat model won’t fit on a single GPU, lilbee spreads it across the ones you have. It sizes each role’s memory with gguf-parser, keeps headroom on every card, and tensor-splits the chat model across the fewest GPUs that fit, with the embedder, reranker, and vision models placed alongside it behind a load-balancing router. This is automatic: ask a question and the model loads split across your cards, answering from your own indexed source. Here a 70B is spread across three consumer cards and answers a mechanic’s question from an indexed car manual, citing the page it came from.
You can also place it by hand. The placement editor pins each role to the cards you choose, previews the fit before anything loads, and applies it live. Ask for a layout that can’t fit and it tells you the exact shortfall instead of failing at load time.
TUI
lilbee (no args) launches a full Textual terminal app: streaming chat with clickable citations, a model bar with searchable pickers and a Search/Chat toggle, a Task Center for background jobs, and screens for the model catalog, settings, and the generated wiki. Type / for the command list; tab completion works everywhere.
Ctrl+P opens the Textual command palette, ? on an empty prompt (or F1 anywhere) toggles the keybinding cheat sheet, /help opens the slash-command catalog. Every action lilbee can take is reachable from one of those three.
Conversations are kept. Reopen an earlier one and it comes back with its history and citations intact.
Fork a conversation after any answer to try a different follow-up. The fork keeps the messages up to and including that answer, and the original stays as it was. Use /fork in the chat, or lilbee sessions fork on the command line. Export a conversation as a markdown document with its sources. Use /export-chat in the chat, or lilbee sessions export on the command line. Both also work over HTTP. See Sessions in the usage guide for the full command set.
Every reel on this page (plus the extras that don’t fit here) is at lilbee.sh/tutorial with long-form captions. Tape sources are in demos-src/ on the gh-pages branch. For commands and settings, see the usage guide.
HTTP Server
The HTTP server exposes a REST API any tool or GUI can hit: search (with SSE streaming), document lifecycle, crawling, model management, configuration. See the REST API reference and the usage guide for setup.
The Obsidian plugin is a GUI built on it: it starts the HTTP server in the background, and every citation opens a Source Preview scrolled to the exact spot. It is an official Obsidian community plugin: install it from Settings then Community plugins inside Obsidian. The plugin README has setup.
Running as a service (optional)
For tools that talk to lilbee’s HTTP REST API (the Obsidian plugin, custom GUIs, anything hitting /api/*), your OS launcher can keep the HTTP server warm so requests skip the cold-start.
Daemon setup per platform
This is the only lilbee surface that benefits from a daemon. The TUI, lilbee chat, the MCP server, and the rest of the CLI load on demand and exit when you close them. No always-on process to babysit.
Pull a chat and embedding model first; all recipes pin the server to 127.0.0.1:42697.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4)
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)
评论