Note: The content and guide here is all dedicated to Delta Lake 101. While the Hitchhiker's Guide is focused on Delta Lake Streaming, we can't start with Streaming before we understand the ins/outs of working with Delta Lake.
For starters, if you are new to using Delta Lake, head over to delta.io to get the million foot introduction. Then we can start working together and tackling Delta 101.
-
Streaming First Steps provides you with a simple journey. You'll learn to create a Delta Lake table using the
DeltaTable
python utility. Then we'll populate the table, and run our first streaming job. (I promise this will be a breeze!). -
After you've completed the
streaming-first-steps.ipynb
, then move onto part-2. 101-dl-streaming.ipynb. We will pick up where we left off and run our first streaming aggregation job.
Note: This is continous work in progress. Please open up an Issue if you find things that seem out of place. Just be nice.
tips and tricks: tbd