Skip to content

Releases: siqueiraa/KaflowSQL

Release v1.0.0

05 Jul 16:35

Choose a tag to compare

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

[1.0.0] - 2025-07-05

🎉 First Major Release!

KaflowSQL v1.0.0 - A production-ready streaming ETL framework for real-time Kafka data joins.

✨ Core Features

  • SQL-Native Streaming Joins - Write familiar SQL to join Kafka topics in real-time
  • High-Performance Processing - 100K+ events/second with sub-millisecond latency (tested on Mac M3 MAX)
  • Intelligent State Management - RocksDB persistence with configurable TTL and S3 checkpointing
  • Smart Emission Control - Configurable emission strategies for data quality vs throughput
  • Multi-Platform Support - Docker images for linux/amd64, linux/arm64

🏗️ Technical Architecture

  • DuckDB Integration - In-memory analytics engine for SQL execution
  • Hash-Based Sharding - 256-way concurrent processing for optimal performance
  • Zero-Copy Processing - Efficient memory usage with pointer-based row storage
  • Avro Schema Support - Full Schema Registry integration with automatic schema evolution

📋 Configuration & Operations

  • YAML Pipeline Definition - Simple, declarative pipeline configuration
  • Flexible State Management - Event TTL with dimension table support
  • Docker-First Deployment - Production-ready containerization
  • Comprehensive Documentation - Complete configuration reference and best practices

🔧 Development & Quality

  • Comprehensive CI/CD - GitHub Actions with automated testing, security scanning, and releases
  • Multi-Platform Builds - Cross-platform binary distribution
  • Security Scanning - Gosec and Trivy vulnerability detection
  • Professional Project Structure - Open-source ready with proper documentation

🐛 Fixed

  • Updated Docker registry configuration for Docker Hub distribution
  • Enhanced pipeline examples with generic use cases

🚀 Performance & Compatibility

  • Cross-Platform Ready - Builds and runs seamlessly on all supported platforms
  • Production Tested - Optimized for high-throughput streaming workloads
  • Docker Optimized - Multi-architecture container images for easy deployment

📦 Distribution

  • Docker Images: siqueiraa/kaflowsql:1.0.0 and siqueiraa/kaflowsql:latest
  • Binary Releases: Cross-platform binaries for Linux, macOS, and Windows
  • Source Code: Full source available on GitHub

[Unreleased]

Added

  • Performance optimizations and monitoring improvements planned for next release

[Previous Versions]

Added

  • Initial streaming ETL framework
  • RocksDB state management
  • DuckDB integration for SQL processing
  • Kafka consumer/producer with Avro support
  • Schema Registry integration
  • TTL-based state cleanup
  • S3 checkpointing for disaster recovery
  • Pipeline configuration system
  • Join processing with hash-based sharding