This is a repo with links to everything you'd ever want to learn about data engineering
-
Updated
Dec 22, 2024 - Jupyter Notebook
#
apachespark
Here are 48 public repositories matching this topic...
Upserts, Deletes And Incremental Processing on Big Data.
bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink
-
Updated
Dec 23, 2024 - Java
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
sql database spark hive hadoop etl pyspark data-engineering spark-streaming data-analysis databricks datalake spark-sql timetravel apachespark etl-pipeline deltalake
-
Updated
Jul 28, 2024 - Python
type-class based data cleansing library for Apache Spark SQL
-
Updated
Jun 23, 2019 - Scala
Code for blog at: https://www.startdataengineering.com/post/docker-for-de/
-
Updated
Apr 29, 2024 - C
SparkSQL.jl enables Julia programs to work with Apache Spark data using just SQL.
-
Updated
Jan 29, 2024 - Julia
FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...
-
Updated
Dec 17, 2024
Delta Lake Examples
-
Updated
Apr 24, 2020 - Jupyter Notebook
Repository for Lab “Distributed Big Data Analytics” (MA-INF 4223), University of Bonn
-
Updated
Aug 11, 2022 - Jupyter Notebook
This repository contains all the projects and labs I worked on while pursuing professional certificate programs, specializations, and bootcamp. [Areas: Deep Learning, Machine Learning, Applied Data Science].
python tensorflow numpy scikit-learn pandas pytorch seaborn recurrent-neural-networks datascience scipy autoencoder image-classification restricted-boltzmann-machine matplotlib machinelearning recommender-system deeplearning ggplot apachespark folium-maps
-
Updated
Oct 13, 2020 - Jupyter Notebook
Trigger spark-submit in Golang. A Go implementation of famous SparkLauncher.java.
-
Updated
Oct 31, 2020 - Go
Connect to SQL Server using Apache Spark
-
Updated
Sep 10, 2016 - Scala
PySpark es una biblioteca de procesamiento de datos distribuidos en Python que permite procesar grandes volúmenes de datos en clústeres utilizando el framework Apache Spark, ofreciendo un alto rendimiento y un conjunto de herramientas integradas para el análisis y manejo de datos a gran escala.
-
Updated
Jun 27, 2023
Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.
emr aws scala big-data bigdata s3 dataframes databricks link-prediction big-data-analytics linkprediction apachespark awsemr
-
Updated
Dec 10, 2019 - Scala
Ce dépôt GitHub contient un document détaillé sur les bases du langage Scala.
-
Updated
Apr 8, 2024
A Capstone Project that covers several aspects of Data Engineering (Data Exploration, Cleaning, Modeling, Pipelining, Processing)
sql bigdata pandas dataset datapipeline datalake dataprocessing dataengineering capstone-project apachespark datacleaning bigdataproject datamodeling datawherehouse dataschema bigdataprocessing
-
Updated
Dec 25, 2022 - Jupyter Notebook
Improve this page
Add a description, image, and links to the apachespark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the apachespark topic, visit your repo's landing page and select "manage topics."