text-splitter 开源项目推荐（59 个）

feyninc /chonkie

🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines

jupediaz /chatgpt-prompt-splitter

ChatGPT PROMPTs Splitter. Tool for safely process chunks of up to 15,000 characters per request

mirth /chonky

文本分块的全神经方法

feyninc /chonkiejs

🦛 CHONK your texts with Chonkie ✨ Type-friendly, light-weight, fast and super-simple chunking library

GiovanniPasq /chunky

A local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.

jparkerweb /semantic-chunking

🍱 Semantically create chunks from large document for passing to LLM workflows

sentencizer /sentencizer

A sentence splitting (sentence boundary disambiguation) library for Go. It is rule-based and works out-of-the-box.

thrivewithai /NinjaSearchWithHumanGPT

An agent with human in the loop that can search the web for information while bypassing bot detection for private sites.

loganliffick /react-spltjs

SpltJS for React

jchunk-io /jchunk

JChunk is a lightweight and flexible library designed to provide multiple strategies for text chunking within Java applications

bhavik-mangla /MDKeyChunker

Header-aware chunking for RAG, plus mk.diagnose: test on your own documents whether to spend on the retriever, a reranker or LLM enrichment. Zero dependencies. Code for arXiv:2603.23533.

thrivewithai /langchain-fixie-marvin

We compared LangChain, Fixie, and Marvin

betcorg /llm-text-splitter

A lightweight TypeScript text splitter for RAG applications

Pavansomisetty21 /RAG-based-Intelligent-Conversational-AI-Agent-for-Knowledge-Extraction-Using-LangChain-Gemini-LLM

In this we implements a Retrieval-Augmented Generation (RAG) based conversational AI agent designed for intelligent knowledge extraction from PDF documents. Leveraging LangChain and Google’s Gemini LLM

pavangupta352 /turnchunk

Chunking that understands who is speaking. Parse any transcript, chunk it without ever splitting a speaker turn. Zero dependencies.

philnash /chunkers

An exploration of text splitting and chunking in JavaScript

thoven87 /text-splitter

Split and analyze text files

Ayubjon /chunklet

Zero-dependency, markdown- and code-aware text chunker for RAG and LLM context windows: token budgets, overlap, and heading breadcrumbs.

NeurosynLabs /ai-prompt-splitter

Free AI Prompt Splitter - Split large documents into chunks for ChatGPT, Claude, GPT-4. Supports PDF, TXT, MD files. Smart token counting & overlap control.

shrutikakapade /RAG-From-Scratch-Complete-RAG-Pipeline-Langchain

A comprehensive repository to learn and implement Retrieval-Augmented Generation (RAG) from scratch using LangChain. It covers the full RAG pipeline including Document Loaders, Text Splitters, Embeddings, Vector Databases, and Retrievers with practical examples and step-by-step explanations.