Apache License 2.0

技术解读

以下内容由 GitRead 中文社区 编辑团队基于项目 README 与官方资料整理, 结合中文开发者的常见使用场景补充技术选型建议。具体以项目官方文档为准。

深度解析 epam/Indigo:化学信息领域的通用工具包

项目背景与痛点

化学信息学是一个涉及到化学数据获取、存储、检索及分析的复杂领域。在这个领域,研究人员经常需要处理大量的化学结构数据,进行反应机理分析、数据库搜索、分子模拟等各种任务。Indigo 是一个由 EPAM 研发的,旨在解决化学信息领域通用工具包缺失痛点的开源项目。它通过提供一个功能强大、高度集成的库,统一了化学结构数据处理的各种需求,使得化学信息学研究变得更加高效便捷。

核心技术揭秘

Indigo 的实现从底层开始注重性能与便捷性。它采用 C++ 实现,封装了化学结构的处理算子与数据结构,为用户提供了一组高效、易用的 API 接口。此外,它还提供了多种语言绑定——Python、Java、JavaScript 等,从而可以跨语言、跨平台地使用。

更重要的是,Indigo 拥有高度模块化的架构,其核心模块可以分别处理各种特定的化学信息学任务。以下是一个简化的数据流程图,展示了 Indigo 核心组件的数据处理流:

graph TD A[化学数据输入] --> B[化合物加载] B --> C[结构处理] C --> D[结构搜索] C --> E{"化学功能?"} E -->|"分析"| F[化学属性计算] E -->|"展示"| G[数据可视化] F --> H[结果输出] G --> H

图中展示了化学数据从输入到处理,再到最终输出的完整处理流程。其中,化学功能判断模块(判断节点 E)指向两个分支:一个是进行化学属性计算,另一个是展示部分数据。

功能亮点与差异

相比同类竞品,Indigo 的核心亮点在于它提供了一个高度集成但又十分模块化的开发工具链。通过使用 Indigo,开发者可以专注于特定领域的应用开发,而不是基于底层计算框架的开发工作。此外,它还集成了结构搜索、分子模拟、反应机理分析等高性能化学信息学工具,极大地提高了工作效率。

应用场景与落地建议

在生产环境中使用 Indigo,开发者需要重点关注性能优化与数据输入格式的兼容性。由于化学数据分析任务往往涉及大量计算,确保工具链的性能是关键。此外,开发者需要根据实际场景合理配置化学数据输入格式,例如选择使用何种化学文件格式(例如 SDF、MOL)来读取化学数据。

该工具包特别适合在以下几个场景下使用:

  • 化学数据库的管理与搜索
  • 化学文献中的化学结构提取与识别
  • 化学反应机理的分析与模拟
  • 化学分子的可视化

综合评价

优势:

  • 高效率的化学结构处理能力,适合大规模化学数据处理。
  • 丰富的功能模块,适配了各种化学信息学应用场景。
  • 支持多种语言,能够跨平台使用,降低了应用门槛。

不足:

  • 由于功能面较广,初始学习曲率较高。
  • 并发处理性能依赖于用户对底层数据结构的理解优化。

总体而言,Indigo 是一个非常优秀的化学信息学工具包,它通过提供一站式的解决方案,将化学信息学研究变得更加高效便捷。无论是研究机构还是企业,都可以借助 Indigo 提高化学数据分析的工作效率和质量。

关于

GitHub 原始描述(英文)

Universal cheminformatics toolkit, utilities and database search tools

描述取自 GitHub 官方 API,保持仓库作者书写的原文。

语言占比

Batchfile
C
C#
C++
CMake
CSS
Dockerfile
HTML
Java
JavaScript
Jupyter Notebook
Makefile
NSIS
PLSQL
PLpgSQL
Pawn
PowerShell
Python
R
SCSS
Shell
TSQL
TypeScript

Release

README

License

EPAM Indigo projects

Copyright © 2009-2022 EPAM Systems, Inc.

Licensed under the Apache License version 2.0

Introduction

This repository includes:

  • Bingo: Chemistry search engine for Oracle, Microsoft SQL Server and PostgreSQL databases
  • Bingo-Elastic: Set of APIs for efficient chemistry search in Elasticsearch
    • Java API. Full README is available here
    • Python API. Full README is available here
  • Indigo: Universal cheminformatics library with bindings to .NET, Java, Python, R and WebAssembly, and the following tools:
    • Legio: GUI application for combinatorial chemistry
    • ChemDiff: Visual comparison of two SDF or SMILES files
    • indigo-depict: Molecule and reaction rendering utility
    • indigo-cano: Canonical SMILES generator
    • indigo-deco: R-Group deconvolution utility

Detailed documentation is available at http://lifescience.opensource.epam.com

Changelog could be found in CHANGELOG.md.

Download

https://lifescience.opensource.epam.com/download/indigo.html

Bindings in public repositories:

Source code organization

Main directory structure layout:

  • api: Indigo API sources
  • bingo: Bingo sources
  • core: Core algorithms and data structures sources
  • third_party: sources for third-party libraries
  • utils: utilities sources

Each project is placed in the corresponding directory with CMakeList.txt configuration file, that does not include other projects. In order to build the whole project with the correct references you need to use CMake configurations from the build_scripts directory.

Develop using dev container

This is an alternative setup solution next to have it on your PC directly. It builds on having the tools in a docker container and VSCode acts as a client.

Prerequisites

  1. Docker Desktop (or docker engine if you prefer)
  2. VSCode

Installation Steps

  1. Clone this repository
  2. Open in VSCode
  3. Follow recommendation to install the Dev Container Extension
  4. Open in Dev Container

The first time takes some minutes but afterwards you can directly develop from VSCode.

Remark for Windows Users: Its a known limitation that on windows the overlay driver is super slow, so if you want to have a fast IDE, clone your container into a volume rather than natively on the client or in your WSL distribution.

Preinstalled build tools

To build the project from the sources, the following tools should be installed:

Required:

  • GIT 1.8.2+
  • C/C++ compilers with C++14 support (GCC, Clang and MSVC are officially supported)
  • CMake 3.4+
  • Python 3.6+

Required to build all targets:

  • JDK 1.8+
  • .NET Standard 2.0+

Required to build Indigo-WASM:

  • Emscripten SDK
  • Ninja
Dependencies:

Python

  • wheel package installed. Command: python -m pip install wheel
  • setuptools version less than 72.0.0 package installed. Command: python -m pip install setuptools==68.0.0
  • waitress package installed (to run backend API test). Command: python -m pip install waitress
  • flasgger package installed (to run backend API test). Command: python -m pip install flasgger
  • psycopg2 package installed (to run backend API test). Command: python -m pip install psycopg2
  • sqlalchemy package installed (to run backend API test). Command: python -m pip install sqlalchemy
  • numpy package installed (to run backend API test). Command: python -m pip install numpy
  • celery package installed (to run backend API test). Command: python -m pip install celery
  • marshmallow package installed (to run backend API test). Command: python -m pip install marshmallow
  • redis package installed (to run backend API test). Command: python -m pip install redis
  • flask_httpauth package installed (to run backend API test). Command: python -m pip install flask_httpauth
  • pyparsing package installed (to run backend API test). Command: python -m pip install pyparsing
  • requests package installed (to run backend API test). Command: python -m pip install requests
  • pillow package installed (required for rendering tests). Command: python -m pip install pillow

On Linux use python3 insted of python. Using virtual environment might be required as well.

Build instruction

On Windows use cmd to run the commands.

  1. Create build folder

    mkdir build
    
  2. Move to build folder

    cd build
    
  3. Run CMake to configure the project with desired options. For instance:

    cmake .. -DBUILD_INDIGO=ON -DBUILD_INDIGO_WRAPPERS=ON -DBUILD_INDIGO_UTILS=ON
    
  4. Build Indigo from console:

    cmake --build . --config Release --target <target name>
    

Replace <target name> with any of the following targets you need:

  • ALL_BUILD (on Windows)
  • all (on Linux)
  • indigo-dotnet
  • indigo-java
  • indigo-python

Build results could be collected from Indigo/dist folder.

‘indigo-python’ target is commonly used.

Run tests

Befor running any test you have to build and install indigo-python

  1. Build indigo-python using --target indigo-python or --target ALL_BUILD(on Windows) or --target all(on Linux). See Build instruction above.
    • On Windows the package should be in ‘Indigo/api/python/dist’ folder
    • On Linux it is located in ‘Indigo\dist’ folder

Package will be named like ‘epam.indigo-<version-arch>.whl’. For instance: epam.indigo-1.29.0.dev2-py3-none-win_amd64.whl

  1. Install package using pip

    • If Indigo package has been already installed, uninstall it with the following command: python -m pip uninstall <path-to-.whl-file> -y
    python -m pip install <path-to-.whl-file>
    

    Replace <path-to-.whl-file> with the right path to .whl package. For instance: python -m pip install ../api/python/dist/epam.indigo-1.29.0.dev2-py3-none-win_amd64.whl

  2. Run integration test

    to run all test

    python api/tests/integration/test.py -t 1
    

    to run tests by mask use test_name

    python api/tests/integration/test.py -t 1 -p test_name
    

    to update all tests “ref/” using -u flag

    python api/tests/integration/test.py -u
    

    to update all tests “ref/” without -u flag

    export INDIGO_UPDATE_TESTS="True"
    python api/tests/integration/test.py
    

    to update a single test’s “ref/” by mask use test_name and -u flag

    python api/tests/integration/test.py -t 1 -p test_name -u
    

To run backend API test:

  1. Build and install indigo-python

  2. Set environment variable by running this command:

    for Linux

    export INDIGO_SERVICE_URL=http://localhost:5000/v2
    

    for Windows in PowerShell

    $env:INDIGO_SERVICE_URL="http://localhost:5000/v2"
    
  3. Run backend service :

    cd utils/indigo-service/backend/service
    
    cp v2/common/config.py .
    
    waitress-serve --listen="127.0.0.1:5000 [::1]:5000"  app:app
    

    you may use any port instead of 5000

  4. Run backend API test:

    python utils/indigo-service/backend/service/tests/api/indigo_test.py`
    

    use -k test_name to run test by pattern.

How to build Indigo-WASM

Build tools prerequisites

  • Git

Make sure git is running from path:

>git --version
git version 2.26.2.windows.1

Make sure python is running from path:

>python --version
Python 3.9.0

Make sure cmake is running from path:

>cmake --version
cmake version 3.18.4

Download corresponding ninja-xxx.zip and unpack to folder on path. Make sure it’s running from path:

>ninja --version
1.10.2
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh

Note: On Windows, run emsdk instead of ./emsdk, and emsdk_env.bat instead of source ./emsdk_env.sh, use cmd instead of powershell.

Get Indigo sources

Clone (or checkout) Indigo repository

>git clone https://github.com/epam/Indigo.git

Build Indigo

For each new session, set environment anew:

>cd emsdk
>./emsdk activate latest

If fresh build:

>mkdir build
>cd build

Now build:

>emcmake cmake .. -DCMAKE_BUILD_TYPE=Debug -G Ninja
>ninja indigo-ketcher-js-test

How to build conda package

Prepare conda build environment as described at https://docs.conda.io/projects/conda-build/en/stable/install-conda-build.html Change directory to ```recipe-conda``, set INDIGO_VERSION to existing Indigo version(already published at PyPi), run conda build: Linux/Mac:

>cd conda-recipe
>INDIGO_VERSION=1.29.0 conda build .

Windows(using cmd):

>cd conda-recipe
>set INDIGO_VERSION=1.29.0
>conda build .

To upload packages: install anaconda-client, login, and upload packet using command provided by conda-build in output

>conda install anaconda-client
>anaconda login
Enter username:
Enter password:
>anaconda upload /conda-build-dir/packet-name

评论

暂无评论