chapi

Chapi

CHAPI (Common Hierarchical Abstract Parser and Information Converter) streamlines code analysis by converting source code from different languages into a unified abstract model, making cross-language analysis and tooling easier.

Chapi 是一个通用层次抽象解析器与信息转换器，它可以将不同编程语言的源代码转换为统一的层次抽象模型， 从而简化跨语言的代码分析与工具构建。

Chapi => Cha Pi => Tea Pi => Tea π => 茶 π. Reference: Tea if by sea, cha if by land.

Chapi (pronounced /tʃɑpi/) can also be read as “XP” in Chinese if you pronounce “X” as “叉”.

Status & language coverage

Language stages

IDL stages

Projects using Chapi

• ArchGuard — An architecture workbench for architecture governance. It can analyze architecture at container/component/code levels, create architecture fitness functions, and inspect system dependencies.
• UnitGen — A fine-tuning data framework that generates datasets from your existing codebase.
• ChocoBuilder — An LLM toolkit for building custom AI assistants.

PS: PRs are welcome — feel free to add your project here.

Kafka streaming API

The chapi-streaming module consumes keyed code-change events and publishes raw AST, project-resolved AST, and diagnostic events through Kafka. Its changelog-backed state stores provide two-pass same-package resolution and eventId idempotency. See the streaming API schema and runbook.

Language information

Tested language versions:

• Java: 8, 11, 17
• TypeScript/JavaScript
• ArkTS: focused .ets and ArkUI application syntax (not compiler or type-checker parity)
• Kotlin
• Rust: v1.60.0
• Python: 2, 3
• Swift: 5, 6 (with typed throws, async/await, actors, ownership modifiers)

Gradle modules (by tier):

// tier 1 languages
":chapi-ast-java",
":chapi-ast-typescript",

// tier 1 model language
":chapi-ast-protobuf",
":chapi-ast-thrift",

// tier 2 languages
":chapi-ast-kotlin",
":chapi-ast-go",
":chapi-ast-python",
":chapi-ast-scala",

// tier 3 languages
":chapi-ast-rust",
":chapi-ast-csharp",
":chapi-ast-c",
":chapi-ast-cpp",
":chapi-ast-swift",
":chapi-ast-arkts",

// others
":chapi-streaming",
":chapi-parser-toml",
":chapi-parser-cmake",

Language families (refs):

• First-class function: https://en.wikipedia.org/wiki/First-class_function
• Algol family: https://wiki.c2.com/?AlgolFamily

Parsing / analysis rules

Chapi scans twice to improve cross-file resolution.

• It helps find data structures in the same package/module.

TypeScript

1. PackageName uses the resolved path. For example, src/grammar/blbla.ts becomes @.grammar.
2. Top-level functions in a file use default as DataStructure.Name.
3. export default Object uses default as FunctionName and belongs to the default data structure.

ArkTS

The experimental chapi-ast-arkts module uses a separate ANTLR grammar derived from the TypeScript module. It supports the common ArkTS application subset, including struct, decorators, @interface, import lazy, ArkUI component blocks, and modifier calls. See the module README for its tested scope and known limitations.

C# notes

• Interpolated string parsing
  • Official grammar: https://github.com/dotnet/roslyn/blob/main/src/Compilers/CSharp/Portable/Generated/CSharp.Generated.g4
  • Related Antlr issue: https://github.com/antlr/grammars-v4/issues/1146
• Import analysis
  • In C#, importing a namespace allows resolving calls inside that namespace.

C

We use https://github.com/shevek/jcpp to preprocess C code.

Kotlin

• warpTargetFullType is required to resolve classes in the same package.

Usage

Add dependencies:

dependencies {
    implementation "com.phodal.chapi:chapi-ast-java:2.5.2"
    implementation "com.phodal.chapi:chapi-domain:2.5.2"
}

Example (Kotlin):

import chapi.domain.core.CodeDataStruct
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.runBlocking
import org.archguard.scanner.core.sourcecode.SourceCodeContext
import java.io.File

class CSharpAnalyser(override val context: SourceCodeContext)

private val client = context.client
private val impl = chapi.ast.csharpast.CSharpAnalyser()

fun analyse(): List<CodeDataStruct> = runBlocking {
    getFilesByPath(context.path) {
        it.absolutePath.endsWith(".cs")
    }
        .map { async { analysisByFile(it) } }.awaitAll()
        .flatten()
        .also { client.saveDataStructure(it) }
}

fun analysisByFile(file: File): List<CodeDataStruct> {
    val codeContainer = impl.analysis(file.readContent(), file.name)
    return codeContainer.Containers.flatMap { container ->
        container.DataStructures.map {
            it.apply {
                it.Imports = codeContainer.Imports
                it.FilePath = file.absolutePath
            }
        }
    }
}

Examples Input & Output

Java source:

package adapters.outbound.persistence.blog;

public class BlogPO implements PersistenceObject<Blog> {
    @Override
    public Blog toDomainModel() {

    }
}

Output:

{
    "Imports": [],
    "Implements": [
        "PersistenceObject<Blog>"
    ],
    "NodeName": "BlogPO",
    "Extend": "",
    "Type": "CLASS",
    "FilePath": "",
    "InOutProperties": [],
    "Functions": [
        {
            "IsConstructor": false,
            "InnerFunctions": [],
            "Position": {
                "StartLine": 6,
                "StartLinePosition": 133,
                "StopLine": 8,
                "StopLinePosition": 145
            },
            "Package": "",
            "Name": "toDomainModel",
            "MultipleReturns": [],
            "Annotations": [
                {
                    "Name": "Override",
                    "KeyValues": []
                }
            ],
            "Extension": {},
            "Override": false,
            "extensionMap": {},
            "Parameters": [],
            "InnerStructures": [],
            "ReturnType": "Blog",
            "Modifiers": [],
            "FunctionCalls": []
        }
    ],
    "Annotations": [],
    "Extension": {},
    "Parameters": [],
    "Fields": [],
    "MultipleExtend": [],
    "InnerStructures": [],
    "Package": "adapters.outbound.persistence.blog",
    "FunctionCalls": []
}

Development

Syntax parsing identification rules:

1. Package name
2. Import name
3. Class / data structure
   1. Structure name
   2. Structure parameters
   3. Function names
   4. Return types
   5. Function parameters
4. Function
   1. Function name
   2. Return types
   3. Function parameters
5. Method call
   1. New instance call
   2. Parameter call
   3. Field call

Build Antlr grammar

1. Install Antlr: brew install antlr
2. Compile grammars: ./scripts/compile-antlr.sh

Data structures

License

@2020 A Phodal Huang’s Idea. This code is distributed under the MPL license. See LICENSE in this directory.