Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.

Scalable and efficient data transformation framework - backwards compatible with dbt.

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, , 20+ connectors

Cloud Native DataOps & AIOps Platform | 云原生数智运维平台

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

📙 Awesome Data Catalogs and Observability Platforms.

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Tenzir is the data pipeline engine for security teams.

DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo

A list of tools for annotating data, managing annotations, etc.

Engine for ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

One framework to develop, deploy and operate data workflows with Python and SQL.

Titan Core - Snowflake infrastructure-as-code. Provision environments, automate deploys, CI/CD. Manage RBAC, users, roles, and data access. Declarative Python Resource API. Change Management tool for the Snowflake data warehouse.

The data-validation toolkit for enhanced dbt (data build tool) PR review

Power BI DevOps & Source Control Tool

Open data platform based on Kubernetes. Scaleph supports SeaTunnel、Flink and Doris backended by SeaTunnel on Flink engine、Flink Kubernetes Operator and Doris operator.