dedupe 开源项目推荐（156 个）

restic /restic

快速、安全、高效的备份程序

dedupeio /dedupe

:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

scinos /yarn-deduplicate

Deduplication tool for yarn.lock files

zinggAI /zingg

Scalable master data management, identity resolution, entity resolution, and deduplication using ML

J535D165 /recordlinkage

A powerful and modular toolkit for record linkage and duplicate detection in Python

Zygo /bees

Best-Effort Extent-Same, a btrfs dedupe agent

nil0x42 /duplicut

从不排序的 MASSIVE 单词列表中去除重复项（用于基于字典的密码破解）。

blakeembrey /free-style

Make CSS easier and more maintainable by using JavaScript

dedupeio /csvdedupe

🆔 Command line tool for deduplicating CSV files

dedupeio /dedupe-examples

🆔 Examples for using the dedupe library

knjcode /imgdupes

Identifying and removing near-duplicate images using perceptual hashing.

Lakshmipathi /dduper

快速的块级带外 BTRFS 去重工具。

kornelski /dupe-krill

A fast file deduplicator

kdeldycke /mail-deduplicate

📧 CLI to deduplicate mails from mail boxes

laktak /chkbit

Check your files for data corruption and run quick file deduplication

jason89521 /daxus

Daxus 是一个面向 React 的服务器状态管理库，可提供对数据的完全控制，从而带来更好的用户体验。

jRimbault /yadf

Yet Another Dupes Finder

zayne-labs /callapi

A lightweight fetching library packed with essential features - retries, interceptors, request deduplication and much more, all while still retaining a similar API surface with regular Fetch.

dssg /pgdedupe

A simple command line interface to the datamade/dedupe library.

jchristn /WatsonDedupe

Self-contained C# library for data deduplication using Sqlite