MIRIX

MIRIX - Multi-Agent Personal Assistant with an Advanced Memory System

Your personal AI that builds memory through screen observation and natural conversation

| 🌐 Website | 📚 Documentation | 📄 Paper | 💬 Discord

---

Key Features 🔥

• Multi-Agent Memory System: Six specialized memory components (Core, Episodic, Semantic, Procedural, Resource, Knowledge Vault) managed by dedicated agents
• Screen Activity Tracking: Continuous visual data capture and intelligent consolidation into structured memories
• Privacy-First Design: All long-term data stored locally with user-controlled privacy settings
• Advanced Search: PostgreSQL-native BM25 full-text search with vector similarity support
• Graph Memory (optional): A provenance hypergraph over episodic and semantic memories in Neo4j, with n-ary facts that link entities through named roles and cite the memory rows they came from
• Multi-Modal Input: Text, images, voice, and screen captures processed seamlessly

Quick Start

Step 1: Backend & Dashboard (Docker):

docker compose up -d --pull always

• Dashboard: http://localhost:5173
• API: http://localhost:8531

Step 2: Create an API key in the dashboard (http://localhost:5173) and set as the environmental variable MIRIX_API_KEY.

Step 3: Client (Python, mirix-client, https://pypi.org/project/mirix-client/):

pip install mirix-client

Now you are ready to go! See the example below:

from mirix import MirixClient

client = MirixClient(
    api_key="your-api-key",
    base_url="http://localhost:8531",
)

client.initialize_meta_agent(
    config={
        "llm_config": {
            "model": "gemini-3.6-flash",
            "model_endpoint_type": "google_ai",
            "api_key": "your-api-key-here",
            "model_endpoint": "https://generativelanguage.googleapis.com",
            "context_window": 1_000_000,
        },
        "embedding_config": {
            "embedding_model": "gemini-embedding-001",
            "embedding_endpoint_type": "google_ai",
            "api_key": "your-api-key-here",
            "embedding_endpoint": "https://generativelanguage.googleapis.com",
            "embedding_dim": 768,
        },
        "meta_agent_config": {
            "agents": [
                {
                    "core_memory_agent": {
                        "blocks": [
                            {"label": "human", "value": ""},
                            {"label": "persona", "value": "I am a helpful assistant."},
                        ]
                    }
                },
                "resource_memory_agent",
                "semantic_memory_agent",
                "episodic_memory_agent",
                "procedural_memory_agent",
                "knowledge_vault_memory_agent",
            ],
        },
    }
)

client.add(
    user_id="demo-user",
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "The moon now has a president."}]},
        {"role": "assistant", "content": [{"type": "text", "text": "Noted."}]},
    ],
    session_id="sess-demo-001",
)

# Include tool activity when you want skills learned from the WORK PROCESS:
# tool errors, retries, and the fix that finally worked are the distiller's
# strongest signals. Pass tool results as role="tool" turns and/or a
# "tool_calls" list on assistant messages (OpenAI shape) — both are preserved
# in the session conversation store.
client.add(
    user_id="demo-user",
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "Deploy the service."}]},
        {"role": "assistant", "content": "", "tool_calls": [
            {"function": {"name": "kubectl_apply", "arguments": "{\"file\": \"deploy.yaml\"}"}}
        ]},
        {"role": "tool", "name": "kubectl_apply", "content": "error: forbidden (missing RBAC)"},
        {"role": "assistant", "content": [{"type": "text", "text": "Fixed the RBAC role and redeployed successfully."}]},
    ],
    session_id="sess-demo-001",
)

memories = client.retrieve_with_conversation(
    user_id="demo-user",
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "What did we discuss on MirixDB in last 4 days?"}]},
    ],
    limit=5,
)
print(memories)

For more API examples, see samples/run_client.py.

Auto-Dream Memory Consolidation

MIRIX also exposes an auto-dream endpoint for explicit memory cleanup and consolidation. Auto-dream reviews existing memories, merges duplicates, resolves stale/conflicting entries where possible, and writes the result back through the memory tools.

REST endpoint:

POST /memory/auto_dream?user_id=demo-user
Content-Type: application/json

{
  "mode": "experience",
  "dry_run": false
}

Supported mode values:

Mode	Memory component(s) reviewed
core	Core memory blocks
episodic	Episodic memories
semantic	Semantic memories
resource	Resource memories
procedural	Procedural memories
knowledge	Knowledge Vault memories
experience	Episodic, Semantic, and Knowledge Vault memories together

Use dry_run: true to fetch counts and inspect what would be processed without applying updates. The optional model field can override the auto-dream agent model for that request. For procedural mode, pass the target meta_agent_id and optionally last_n_sessions to distill recent session experiences.

Python client example:

result = client.auto_dream(
    user_id="demo-user",
    mode="procedural",
    meta_agent_id=meta_agent.id,
    last_n_sessions=3,
    dry_run=False,
)
print(result)

Graph Memory

MIRIX can index episodic and semantic memories into a provenance hypergraph stored in Neo4j. Entities become anchors, each extracted fact becomes an n-ary frame whose edges name the role each anchor plays, and every frame cites the memory rows it came from. With graph memory on, episodic and semantic retrieval enters through the graph, and Auto-Dream merges duplicate anchors on the graph instead of rewriting stored memories.

Graph memory is off by default. To turn it on:

1. Start Neo4j 5.23 or later. docker compose --profile graph up -d starts Neo4j 5.26 with APOC; first set MIRIX_NEO4J_PASSWORD (at least 8 characters) in .env, because there is no default password.
2. Install the Neo4j driver. It is in requirements.txt; after pip install -e ., run pip install "neo4j>=5.20,<6".
3. Set MIRIX_ENABLE_GRAPH_MEMORY=true and MIRIX_NEO4J_PASSWORD to the password your Neo4j uses. MIRIX_NEO4J_URI and MIRIX_NEO4J_USER default to bolt://localhost:7687 and neo4j.

With graph memory on, POST /memory/auto_dream refines the graph only unless the request sets graph_only to false. Evaluation settings and commands are in evals/GRAPH_MEMORY.md.

Upgrading an Existing Database

Fresh databases get their full schema automatically at startup. Existing databases require manual migrations when upgrading to a version that alters tables — startup creates missing tables but never adds columns to existing ones. If migrations are pending, the server refuses to start and names the exact scripts to run.

For the skill-based procedural memory release, run against your database (in this order):

psql "$MIRIX_PG_URI" -f scripts/migrate_procedural_to_skill.sql
psql "$MIRIX_PG_URI" -f scripts/migrate_add_message_session_id.sql
psql "$MIRIX_PG_URI" -f scripts/migrate_add_message_session_id_phase2.sql   # outside a transaction (CREATE INDEX CONCURRENTLY)
psql "$MIRIX_PG_URI" -f scripts/migrate_add_conversation_message.sql
psql "$MIRIX_PG_URI" -f scripts/migrate_add_conversation_message_phase2.sql # outside a transaction
psql "$MIRIX_PG_URI" -f scripts/migrate_add_skill_experience.sql
psql "$MIRIX_PG_URI" -f scripts/migrate_add_agent_trigger_state.sql
psql "$MIRIX_PG_URI" -f scripts/migrate_backfill_procedural_scope.sql   # backfill filter_tags['scope'] from each skill's owning client

For the graph-memory release, which adds five provenance columns, also run:

psql "$MIRIX_PG_URI" -f scripts/migrate_add_graph_provenance_columns.sql

Each script is idempotent where possible and documents its own preconditions in its header. Run them once, then restart the server.

License

Mirix is released under the Apache License 2.0. See the LICENSE file for more details.

Contact

For questions, suggestions, or issues, please open an issue on the GitHub repository or contact us at [email protected]

Join Our Community

Connect with other Mirix users, share your thoughts, and get support:

💬 Discord Community

Join our Discord server for real-time discussions, support, and community updates: https://discord.gg/S6CeHNrJ

🎯 Weekly Discussion Sessions

We host weekly discussion sessions where you can:

• Discuss issues and bugs
• Share ideas about future directions
• Get general consultations and support
• Connect with the development team and community

📅 Schedule: Friday nights, 8-9 PM PST 🔗 Zoom Link: https://ucsd.zoom.us/j/96278791276

📱 WeChat Group

You can add the account ari_asm so that I can add you to the group chat.

Acknowledgement

We would like to thank Letta for open-sourcing their framework, which served as the foundation for the memory system in this project.