技术解读

以下内容由 GitRead 中文社区 编辑团队基于项目 README 与官方资料整理, 结合中文开发者的常见使用场景补充技术选型建议。具体以项目官方文档为准。

深度分析:feng4887/TopoData

项目背景与痛点

在工业自动化领域,PLC(可编程逻辑控制器)和 DCS(分布式控制系统)扮演着核心角色,能够控制和监控工厂中的各种设备与设备之间的通信。然而,一个复杂的企业环境中,面临的问题不只是设备之间的通信,还需要一个强大的系统来采集数据、进行存储、并在必要时通过多种协议(例如 HTTP 和 MQTT)推送数据。此外,这些系统还需要灵活的配方定义功能来满足不同生产任务的需求。

在这种背景下,TopoData 应运而生。它不仅提供了数据采集和存储的能力,还整合了 Python 脚本功能,让用户可以自定义逻辑处理,并通过 HTTP WebAPI 和 MQTT 协议提供了灵活的数据读写接口。这些特性使得 TopoData 成为一个全能的控制系统,满足工业自动化中的多重需求。

核心技术揭秘

技术架构分析

TopoData 的实现结合了多项关键技术栈,并精心设计了一整套数据流来支持其核心功能。以下是一个简化版的技术架构图,展示其内部的数据流处理过程:

graph TD A[用户请求] --> B[API 网关] B --> C{"命令分发"} C -->|"数据获取"| D[数据采集层] C -->|"命令执行"| E{"逻辑执行"} E -->|"数据处理"| F[Python 脚本] E -->|"数据交互"| G[MQTT 接口] E -->|"状态同步"| H[HTTP WebAPI] D --> I[数据缓冲] I --> J[数据存储] I --> K{"数据同步"} K -->|"状态上报"| H K -->|"更新事件"| F

从上图可以看出,TopoData 主要由几大模块构成:API 网关、命令分发器、数据采集层、Python 脚本处理、数据存储等。各个模块之间通过精细设计的接口进行通信,确保数据在整个系统中高效流转。

数据采集层主要负责从 PLC 和 DCS 采集数据。数据获取后,首先会被缓冲处理以避免实时处理的压力,并进行初步过滤和格式化。之后,数据会被分批存储到持久层。

逻辑执行层面则是通过 Python 脚本运行自定义的业务逻辑。这种设计不仅能够提高系统的灵活性,同时还能让用户根据自身需求进行扩展。

数据同步逻辑保证所有的数据操作在应用状态和存储之间保持一致性,通过 HTTP 和 MQTT 接口将内部状态同步给外部系统。

实现原理与关键技术

  1. 数据采集与存储:数据采集采用异步机制,确保系统在高负载情况下仍能有效处理实时数据流。对于持久层,可以采用多种数据库进行存储,以满足不同的数据存储需求。数据流的处理是模块化和可插拔的,易于扩展和维护。

  2. Python 脚本能力:通过 Python 脚本,开发人员可以实现自定义业务逻辑,大幅增强了系统的灵活性。此外,Python 脚本的执行无需重新编译,支持热更新,契合工业场景中的动态需求。

  3. WebAPI 和 MQTT 协议支持:HTTP WebAPI 和 MQTT 提供了灵活的数据接口,适用于不同的协议要求和场景需求。WebAPI 便于快速集成,而 MQTT 适用于低带宽、高延迟或昂贵网络环境中的设备通信。

功能亮点与差异

TopoData 最引人瞩目的亮点在于它对 Python 脚本的支持,赋予用户便捷的自定义业务逻辑开发能力。此外,灵活的 WebAPI 和 MQTT 接口提供了前所未有的灵活性,易于在不同的业务环境中做到无缝集成。

与其他竞品相比,TopoData 的一大杀手锏是在数据采集、存储和处理方面拥有了更强的灵活性和扩展性。它不仅能够满足传统的工业数据采集需求,还通过灵活的自定义功能满足高度个性化的生产任务需求,特别是在需要频繁调整工艺参数的场景下,表现尤为突出。这使其既能适应小规模系统的快捷部署,也能在大规模、复杂生产环境中承担重任。

应用场景与落地建议

TopoData 可以应用于各种工业自动化系统,特别适用于生产线监视、设备状态监控、数据记录和远程管理等场景。例如,在一个装备有多个传感器和控制器的工厂中,TopoData 可以用于统一管理所有设备的数据流和命令执行。以下是几个使用 TopoData 的建议:

  1. 确保安全和数据的隐私:在生产环境中部署 TopoData 时,必须严格控制访问权限,并确保数据传输的安全性。

  2. 网络稳定性:注意到 TopoData 依赖网络进行数据通信,因此在部署前必须确保网络环境的稳定性,特别是在使用 MQTT 接口时,需优化配置提升传输效率。

  3. 扩展性和可维护性考虑:考虑到可能增加新的数据源或交换协议,TopoData 的模块化设计使其易于扩展。充分利用其灵活性,在现有基础上不断优化和完善系统功能。

综合评价

总体来看,TopoData 是一款设计精良、功能齐全的 C# 应用,尤其适用于工业自动化领域。通过利用 Python 脚本、WebAPI 和 MQTT 接口等特性,TopoData 在灵活性、扩展性和可维护性方面表现优秀。唯一的缺点在于复杂的部署配置要求一定的技术背景,但其内置的灵活性和适用性确保了它能在广泛的工业场景中发挥巨大作用。

总结其优点与不足,TopoData 无疑是工业自动化控制领域中一个不容忽视的选择。

关于

GitHub 原始描述(英文)

TopoData is a C# application, provides PLC/DCS data acquisition , storage and python script capabilities , along with HTTP WebAPI、Mqtt interfaces for data read/write interfaces. Provides flexible recipe definition functionality, allowing users to select different recipes and download parameters to PLCs or devices.

描述取自 GitHub 官方 API,保持仓库作者书写的原文。

简要分析

热度分
161
价值分
91
活跃状态
活跃

语言占比

C#
Shell

Release

贡献者

README

HiTopo

V1.1 Readme

TopoData provides data acquisition and storage capabilities for the industrial automation sector, along with data read/write interfaces for third-party software, aiming to help system integrators bridge the communication gap between IT and OT.

The outstanding features of TopoData software are as follows:

Supports Siemens Profinet protocol and OPC UA protocol for communication with devices.
 Provides HTTP WebAPI interfaces for IT systems to read and write PLC or device data.
Offers system diagnostics functionality, allowing engineers to view PLC communication status, read/write data, and download recipe parameters.
Provides relational database data storage functionality with a flexible storage mechanism; data storage triggering conditions can be configured as cyclic storage, event-based storage, or expression-based storage.
Flexible definition of binding relationships between database table structures and communication points; engineers do not need to understand SQL statements, as the system can generate database table structures with one click.
Database support includes three options: Microsoft SQL Server, SQLite, and PostgreSQL.
Provides flexible recipe definition functionality, allowing users to select different recipes and download recipe parameters to PLCs or devices.
 Historical data can be exported to Excel.
 HMI Data acquisition and storage models for industry software engineers.

1 Installation

1.1 Installation Environment

The software is compatible with Windows 7 and above operating systems.
The software is developed based on DotNet 8. Therefore, before using it, the following components need to be installed:

 windowsdesktop-runtime-8.0.11-win-x64.exe
 dotnet-runtime-8.0.11-win-x64.exe
 aspnetcore-runtime-8.0.11-win-x64.exe

2 System configuration

2.1 WebAPI Configuration

Third-party IT software can access real-time data of the data acquisition channel through the HTTP WebAPI and also perform write operations.

2.2 Database configuration

Database Type: A drop-down list of database types, allowing you to select the type of database to store, including Microsoft SQL Server, PostgreSQL, and MySQL.
Database keep Time: In months, with a minimum of 1 month. Data older than this period will be automatically deleted by the service software, and only data within this period will be retained.
1 SQL Host: The address of the server where the database is located.
Database: The name of the database.
Database Storage Location: Only useful when creating an SQL Server database. For PostgreSQL and MySQL, refer to the corresponding manual for data storage locations. Users select a folder and click the “DB Create” button to create the database (if a remote database is configured, users need to create the same folder structure locally and remotely when selecting. If a local location is selected, the remote database will be stored in the corresponding location).
 Username: The database user.
Password: The database login password.

Port: The database port.
Local Log: Logs will be stored in the Logs folder and are mainly used for software debugging or finding system bugs. It is disabled by default.
Create DB Button: Create the database based on account, password, port, and other information.
Test DB Button: Test the database connection status based on the configuration.
 Save Button: Save configuration.

2.3 Automatic Start

By creating shortcuts in the startup folder, the software can be made to start automatically upon windows booting.

3 Equipment Management

3.1 Simens Profinet Equipment

User select “Siemens Profinet” Driver Type, this driver provides communication functionality with Siemens S7-300/400 and S7-1200/1500 PLCs.

 Special Settings for S7-1200 and S7-1500 in TIA Portal Software

√ The CPU must be configured to allow PUT/GET communication access from remote partners (the interface is similar to the following).

 If you need to access DB blocks, the DB blocks must be configured for non-optimized address access:

 配置点表

TopoData suportated:

numTopoData DataTypeTIA DataTypeDescription
1IntDint
2BoolBoolboolean
3FloatFloat32Float 32
4StringStringstring
5WstringWstringUnicode string
6WORDWORDint 16

As shown in the figure below, TopoData supports configuring the point table via Excel import and export. The first column is the variable name, which will be used later for database storage and external interfaces; the second column is the PLC point address (the address naming is the same as in TIA Portal, e.g., DB301.wstring12.10, where 301 is the DB block number, 12 is the absolute address of the string, and 10 is the string length); the third column is the data type; the fourth column is the unit of the point (if no unit is required, use another character as a placeholder, preferably not left blank); the fifth column is the remark or description of the variable name.

Notes:

For the first-time configuration, users can click the “Export” button. The software provides a sample address format, which users can edit based on, then import and save.
To improve communication efficiency, it is recommended to place data communication points in one or as few DB blocks as possible, with data locations as adjacent as possible.
The current driver only supports read/write operations for DB blocks; other memory areas are not supported at this time.
Bool-type data does not support write operations.

3.2 OPC UA Equipment

User select “OPCUA” Driver Type, this driver provides communication functionality with devices via OPC UA communication type.

“Get Endpoint” button: After correctly filling in the URL, click the “Get Endpoint” button. The system will enumerate all Endpoints. The user selects one and clicks the “Save” button to save it.
1 The “Brows Tag” button: A dialog box pops up for selection. Right-click on the parent node and select “Get all subItems” from the context menu to add sub-level all Tags, or select the yellow icon node individually and choose “Get this item” to add the current OPC UA Tag.

Import and export tags

As shown in the figure below, TopoData also supports configuring the point table through Excel import and export. The first column is the variable name, which will be used later for database storage and external interfaces; the second column is the UA point address (must start with “ns”); the third column is the unit of the point (if no unit is required, use another character as a placeholder, preferably not left blank); the fourth column is the remark or description of the variable name.

Note :

For the first time, users can click the “Export” button. The software will provide an example address format, which users can edit based on, then import and save.
The “On-Off switch” allows users to enable or disable the device communication function.

4 DataTable storage

This functional module is mainly used for data storage configuration. Users can utilize a fixed database table structure or create their own, bind corresponding PLC communication points, and store the data.

Storage Information

Task: Storage Task

DB Table: user defined Database table name

Equipment: equipment name

Switch button: enable or disable database storage

Time scale table checkbox: If this option is not selected, data will be stored in the user-defined table structure, with the table name taken from the “DB Table” text box; if this option is selected, data will be stored in the table “tb_au_pubHisvalues”.

数据存储规则

Consecutive Acquisition: As shown in the figure below, select the " Consecutive Acquisition" radio button to enable the system to continuously collect data according to the acquisition cycle. Users need to fill in the acquisition cycle and a bool-type trigger condition data acquisition point. When the bool value is true, the software performs table data storage; when false, the system does not store data. To achieve more complex acquisition and storage rules, please check the " Expression" checkbox, click the “Expression” button, and a dialog box will pop up for configuring the expression conditions.

Rising Edge Acquisition: As shown in the figure below, select the “Rising Edge Acquisition” radio button to enable table data storage when the system detects a bool value of On/True. Users need to configure a bool-type trigger condition point.

To achieve more complex acquisition and storage rules, please check the " Expression" checkbox, click the “Expression” button, and a dialog box will pop up for configuring the expression conditions.

Use Expression: Expressions can be used in both Rising Edge and Continuous Acquisition modes. An expression is a formula that combines multiple variable conditions to determine the data storage condition.

Users check the " Expression" checkbox and click the “Expression” button to pop up the Expression Editor dialog. First, users click the plus button to the right of the parameter list in the middle to add one or more internal variables for the expression. Then, edit the expression in the expression text box, click the red checkmark button to verify if the expression is correct. If correct, click the Save button to save the expression.

Data Collection

As shown in the figure below, users can custom-configure the storage data table structure, where each data column corresponds to a device communication point. The system stores a set of real-time data into this table according to the acquisition rules configured above.

Batch Tag

This configuration is optional. If a batch number communication point is configured, the system will assign work order (WO) information to the corresponding reserved WO field in the table based on this point. This communication point must be in string format.

Batch Tag

5 Recipe Management

Users can define multiple sets of recipe download parameters, where each recipe can define different download parameters as well as different default values.

5.1 Button action

Add,Delete and Save configuration buttons.

5.2 Basic information

Recipe name and description define as well as enable configuration.

5.3 Recipe parameters

Table for recipe parameters as well as configuration buttons.

6 Parameter Diagnose

If all modules have been correctly configured according to the requirements in Chapters 3, 4, and 5, click the ‘Start’ button at the top to switch to the ‘Parameter Diagnose’ module for real-time monitoring and operation of the equipment.

1 The module provides filtering functionality by TagName.
The module provides recipe parameter download functionality.
Double-click the communication point to pop up a dialog box, where users can enter the value they want to write and send it to the controller.

7 Historian query

Data stored in the relational database can be queried through the Historical Query module and exported to Excel.

If ‘DataTable’ is selected, the queried data comes from the database table ‘ProcessData’ configured in Chapter 5; if ‘Time Scale Table’ is selected, the queried data comes from the database table ‘tb_au_pubHisvalues’.

8 WebAPI Read and Write

If external WebAPI interface functionality is required, the WebAPI service address must first be configured in Section 2.1

8.1 WebAPI TagName

In the WebAPI and Parameter Diagnostics modules, the TagName is equivalent to ‘Data Acquisition Channel $^ +$ Point Name’ in the Device Management module.

8.2 Read Tags

WebAPI Post:

http://127.0.0.1:8093/api/Data/GetRealtimeValues

Content-Type: application/json

The input parameter is a JSON string array.:

Request Body:

[ "R0021.alm1", "R0021.pH", "R0021 pkg_Bucket"] 

8.3 Write Tags

WebAPI Post:

http://127.0.0.1:8093/api/Data/WriteRealtimeValues

Content-Type: application/json

The input parameter is a JSON string array.

Request Body:

[ { "CanneIID": "R0021", "ValueName": "R0021.alm1", "value": 1 } ] 

8.4 Download Recipe

WebAPI Post:

http://127.0.0.1:8093/api/Recipe/DownloadRecipe

Content-Type: application/json

The input parameter is a JSON string array.

Request Body:

“多肽配方”

9 License

MIT License

Copyright © [2026] [Topotech Team]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

评论

暂无评论