Logstash - transport and process your logs, events, or other data
-
Updated
Dec 19, 2024 - Java
#
etl-framework
Here are 194 public repositories matching this topic...
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
python rust streaming real-time kafka etl machine-learning-algorithms stream-processing data-analytics dataflow data-processing data-pipelines batch-processing pathway iot-analytics etl-framework time-series-analysis
-
Updated
Dec 23, 2024 - Python
The open source high performance ELT framework powered by Apache Arrow
go kubernetes github-api bigquery aws data google sql etl azure gcp data-engineering data-analysis data-integration data-collection elt etl-framework cspm airbyte attack-surface-management
-
Updated
Dec 23, 2024 - Go
Flow-based programming for JavaScript
-
Updated
Apr 7, 2024 - JavaScript
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
python data-science machine-learning etl pandas orchestration data-engineering data-analysis software-engineering feature-engineering dataframe hacktoberfest dag lineage etl-framework etl-pipeline rag mlops llmops
-
Updated
Dec 17, 2024 - Jupyter Notebook
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
python airflow spark apache-spark scheduler s3 data-engineering data-lake warehouse redshift data-migration livy etl-framework apache-airflow emr-cluster etl-pipeline etl-job data-engineering-pipeline airflow-dag goodreads-data-pipeline
-
Updated
Mar 9, 2020 - Python
This repository is a getting started guide to Singer.
-
Updated
Sep 3, 2024 - Makefile
Making data lake work for time series
-
Updated
Aug 21, 2024 - Python
A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton
python data-science machine-learning etl numpy pandas data-engineering data-platform software-engineering feature-engineering dataframe dag hamiltonian etl-framework hamilton featurization etl-pipeline stitch-fix
-
Updated
Jul 3, 2023 - Python
SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).
real-time offline high-performance apache data-integration sql-engine data-pipeline etl-framework seatunnel
-
Updated
Dec 19, 2024 - Java
A simplified, lightweight ETL Framework based on Apache Spark
-
Updated
Jan 24, 2024 - Scala
Extract, Transform, Load: Any SQL Database in 4 lines of Code.
-
Updated
May 23, 2019 - Python
Knowledge Graph Toolkit
-
Updated
Oct 15, 2023 - Jupyter Notebook
A tool for building feature stores.
-
Updated
Dec 5, 2024 - Python
A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
aws data-science data aws-s3 redshift etl-framework aws-glue aws-lake-formation lakehouse lakeformation
-
Updated
Dec 23, 2024 - Python
Bender - Serverless ETL Framework
-
Updated
Dec 19, 2023 - Java
Configurable Extract, Transform, and Load
mysql files elasticsearch sql-server etl solr sqlite excel postgresql data-warehouse sqlce denormalize etl-framework ssas
-
Updated
Oct 19, 2024 - C#
Improve this page
Add a description, image, and links to the etl-framework topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the etl-framework topic, visit your repo's landing page and select "manage topics."