Home
Kafka-Penguin
An easy-to-use, lightweight KafkaJS library for message re-processing.
Table of Contents
About
Kafka-Penguin is an easy-to-use, lightweight KafkaJS plugin for message re-processing. It provides developers with three strategies for setting up message re-processing: FailFast, Ignore, and Dead Letter Queue.
The package allows developers to build event-driven applications with dedicated “fail strategies” modeled after best practices in the field. This in turn allows developers to effectively address bugs in development and deploy more fault-tolerant systems in production.
This package is meant to work in conjunction with with KafkaJS. For more information on KafkaJS, check out Getting Started with KafkaJS.
Getting Started
Install Kafka-Penguin as an npm module and save it to your package.json file as a dependency:
npm install kafka-penguin
Once installed it can now be referenced by simply calling require('kafka-penguin');
Example
All Kafka-Penguin needs is a KafkaJS client to run. Start by passing the client for your preferred strategy and Kafka-Penguin will create bespoke consumers, producers, and admins with built-in functionality to execute the chosen strategy. On the surface, you implement your application exactly as you would with KafkaJS.
/* eslint-disable no-console */
import { FailFast } from 'kafka-penguin';
const exampleClient = require('./clientConfig.ts');
// Set up the preferred strategy with a configured KafkaJS client
const exampleStrategy = new FailFast(2, exampleClient);
// Initialize a producer or consumer from the instance of the strategy
const producer = exampleStrategy.producer();
const message = {
topic: 'wrong-topic',
messages: [
{
key: 'hello',
value: 'world',
},
],
};
// Connect, Subscribe, Send, or Run virtually the same as with KafkaJS
producer.connect()
.then(() => console.log('Connected!'))
// The chosen strategy executes under the hood, like in this send method
.then(() => producer.send(message))
.catch((e: any) => console.log('error: ', e.message));
Strategies
Dive in deeper to any of the strategies for set up, execution, and implementation.
FailFast
Ignore
Dead Letter Queue
Contributors
Ausar English @ausarenglish
Kushal Talele @ktrane1
Timeo Williams @timeowilliams
Ziyad El Baz @zelbaz946
License
This product is licensed under the MIT License - see the LICENSE.md file for details.
This is an open source product. We are not affiliated nor endorsed by either the Apache Software Foundation or KafkaJS.
This product is accelerated by OS Labs.
评论