tokenizer-parser 开源项目推荐（52 个）

risesoft-y9 /Data-Labeling

数据标注是一款专门对文本数据进行处理和标注的工具，通过简化快捷的文本标注流程和动态的算法反馈，支持用户快速标注关键词并能通过算法持续减少人工标注的成本和时间。数据标注的过程先由人工标注构建基础，再由自动标注反哺人工标注，最后由人工标注进行纠偏，从而大幅度提高标注的精准度和高效性。数据标注需要依赖开源的数字底座进行人员岗位管控。

ikskuh /parser-toolkit

A toolkit that makes it easier to write recursive-descent parsers in Zig.

luauminati /LuaXen

Lua Compiler, (De)Obfuscator, Minifier, Beautifier, And more

ehwan /C-language-Parser-In-Rust

C language lexer & parser & virtual interpreter from scratch

recp /xml

🔋 In-place lightweight XML parser

melchisedech333 /antlr4-experiments

🔧 My studies on context-free grammar, using ANTLR4 (C++) to generate the parser files. Some basics are developed, such as token processing, recursion, variable definition, array processing, Abstract Syntax Tree (AST) manipulation, UNICODE support, and error handling.

zbroderson /HumanTimeParser

Simple to use parser capable of parsing a usable time object from human input

aboss123 /Ice-Programming-Language-Submission

This is a short and modern JIT compiler that transform source text, into LLVM IR bytecode that executes machine code at runtime. This project was developed at the hths.hacks() hackathon against more 250+ participants internationally and was placed as a winner. Among the winners, my project was the only one developed solo.

melchisedech333 /lex-yacc-experiments

🔧 My studies involving context-free grammar analysis. The analyzers were built using familiar tools such as YACC, Lex and Bison. Topics covered include token filtering, simple variable manipulation, and arrays.

TheDoctorHouse /Parser

A simple programming language interpreter, lexer, and parser written in C#.

spydaz /AI_ZX81

A Basic Experiment in Parser and Compilers and Stack VM . A basic stack based CPU with Assembly language and basic commands. A basic programming Languge Parsed to Tokens to e parsed to expressions to be compiled to assembly code to be executed on the virtual CPU... Also to be used to Parse English grammar to make abstract syntax trees.

daireto /odata-v4-query

A lightweight, simple and fast parser for OData V4 query options supporting standard query parameters. Provides helper functions to apply OData V4 query options to ORM/ODM queries such as SQLAlchemy and Beanie.

jaiva-lang /jaiva

An esoteric functional programming language written in Java (hobby project)

Rajspeaks /Machine-Learning-approach-to-Bengali-Corpus-Tokenization-Stemming-POS-Tagging-using-BNLTK

Machine Learning approach to Bengali Corpus POS Tagging using BNLTK. This is an experimenting project under the mentorship of Prof. Sandipan Ganguly, HIT-K.

stcjs /flkit

JS/HTML/CSS 工具包（Tokenizer、Parser）支持模板语法

DuffsDevice /tiny-parser

Write use-case specific parsers within minutes!

zavvdev /elise-lang

A domain-specific language for schema-driven data processing, with ahead-of-time compilation to type-optimized bytecode. Write once, run against any conforming dataset.

MitchellGray100 /Critter-World-README

A README for my private CS 2112 Critter World Project

jackr276 /Simple-Pascal-Like-Language-Interpreter

An interpreter for a custom-made, Pascal-Like Programming Language

nousantx /nguraijs

A small and lightweight library for string tokenizing