2.8.8

Added:

luigi

• Expandable Namespace Folders for the Visualiser Sidebar #2716
• Added new companies to the luigi users list: #2730 #2747

luigi.contrib

• Enable Overriding Poll Interval for Kubernetes Jobs #2724

Fixed:

luigi

• Code example correction #2754

Changed:

luigi

• Update release process #2727
• Change GET request to POST requests in luigi/rpc #2732
• Fix SendGrid email API documentation. #2745

2.8.7

Added:

luigi

• Add check_complete_on_run to optionally mark tasks as failed if complete() is false when run finishes (#2710)
• Add section "Running Luigi on Windows" to docs (#2720)
• Add Giphy to list of companies using Luigi to docs (#2713)

luigi.contrib

• Add DropboxTarget for luigi (#2696)

Fixed:

luigi

• UI: Fix time graph - y axis to account for timezones. (#2711)

Changed:

luigi

• Bump dependencies used by SendGrid integration. (#2715)
• Update copyright year in LICENSE (#2723)

luigi.contrib

• Make RedisTarget compatible with redis-py >= 3 (#2722)

2.8.6

Fixed:

luigi

• fix calling method on wrong object #2709
  This fixes #2704.

2.8.5

Fixed:

luigi

• (Doc) Fix example of "summary_length" #2700
• Fix __init__ error when using TOML config #2702
• add callback to metric collector #2704

luigi.contrib

• Fix BigQueryTarget parsing in beam_dataflow module #2705

Changed:

luigi.contrib

• aws batch : job queue as parameter #2689

2.8.4 (BROKEN)

This release is broken due to #2628 .

Added:

luigi

• Added support for a detailed LuigiRunResult instead of a plain Boolean (#2630)
• Add worker option 'max_keep_alive_idle_duration (#2654)
• Added worker-id commandline parameter (#2655)

luigi.contrib

• Add support for specifying kubernetes namespace (#2629)
• Add a Task wrapper for MicroSoft OpenPAI (#2531)
• Provide automatic URL tracking for Spark applications (#2661) (#2669)
• add beam_dataflow_task to luigi/contrib (#2675)

Both

• Add Prometheus contrib for monitoring purpose (#2628)

Fixed:

luigi

• setup.py: Support older setuptools (<=20.1.1) (#2623)
• Fix broken aws tests (#2658)
• Accept pathlib based path as argument for LocalTarget (#2548)
• Fix durations in D3 graph (fixes #2620) (#2624)
• Import collections ABCs from collections.abc, not collections (#2683)
• Configuration documentation: remove deprecated/wrong [core]max_reschedules entry (#2692)

luigi.contrib

• fixes #2223 HdfsTarget is not working with snakebite (#2572)
• Add port field for PostgresQuery (fixes #2625) (#2627)

Both

• Fix flake errors after moving to python 3 (#2695)

Changed:

luigi

• Simplify implementation of temporary_path() (#2652)
• Prevent range-tasks from autoloading (#2656)
• Replace Python MapReduce example with Spark example. (#2668)
• Minor improvements to single-worker-timeout support (#2667)
• Require at least python-dateutil version 2.7.5 instead of only 2.7.5 (fixes #2662) (#2679)
• RangeMonthly should deal with whole months (#2666)
• Reconcile underscore/dash config style handling (#2688) #2691

luigi.contrib

• Add autodetect parameter to BigQueryLoadTask (#2363) (#2575)

2.8.3

Added:

luigi

• Add BaseTIS to the company list #2607
• Add Hopper to the company list #2614
• give a few default values to opts when setting up logging #2612
• Add range functionality for monthly cadence. #2601

luigi.contrib

• Added port to PostgresTarget #2615
• Support for Azure Blob Storage Target #2585
• Add Datadog contrib for monitoring purpose #2434

Fixed:

luigi

• Docs: Fixed a mistake with @inherits syntax in luigi/util.py #2613
• Check type of column before migrating schemas for task db history for postgres dialect (fixes #2563) #2564

luigi.contrib

• S3: Fix call to message from TypeError not working with Python 3.6 #2617
• Use proper API call in bigtable.py's make_dataset #2618
• Sqla: Fix the table name when reflect is True in sqla.CopyToTable (fixes #2604) #2605

Changed:

luigi.contrib

• Changed to buffered reads when using GCSTarget #2588

2.8.2 (Hotfix)

• fix logging setup in Python 2.7 (#2593)

2.8.1(broken)

Note: Broken due to a runtime error in LuigiConfigParser. See #2592.

Added:

luigi

• Add some docs to interface.run #2582
• Configure logging via TOML config #2483

luigi.contrib

• Added port property to CopyToTable #2561
• Make it so we can do from luigi.contrib.hdfs import HdfsFlagTarget #2594
• contrib: Add ExternalDailySnapshot #2591

Fixed:

luigi

• (docstring) Update task.py #2589
• Docs: Fixed "Github" to fit to the rest of the doc #2596
• Fix inspect.getargspec() DeprecationWarning in PY3 #2579

luigi.contrib

• Fix Travis Moto Test Failures #2586
• Reduce TravisCI Test Runtime #2541

Changed:

luigi

• Make Worker parameter task_process_context an OptionalParameter #2574

luigi.contrib

• S3Client improvements #2569

2.8.0

This is a minor version bump, due to:

• Dropping Python 3.4 and 3.5 from CI, which means no automated tests to ensure compatibility for those versions
• [Security Patch] CORS being disabled by default. A new section of configuration [cors] is introduced to enable custom settings. For details, refer to user group topic: https://groups.google.com/forum/#!topic/luigi-user/ZgfRTpBsVUY

Added

luigi:

• Add Python 3.7 compatibility (#2466)
  This also drops 3.4 and 3.5 from CI.
• Interpolate environment variables in .cfg config files (#2527)

luigi.contrib:

• Add CopyToTable task for MySQL (#2553)
• Add HdfsFlagTarget (#2559)

luigi.contrib:

Fixed

luigi:

• Fix ReadTheDocs build (#2546)
• Make capture_output non-positional in ExternalProgramTask (#2547)

luigi.contrib:

• Fix S3Client's _path_to_bucket_and_key to support keys with question marks (#2534)
• Fix S3Client.remove - add max batch size (#2529)
• Small fix to logging in contrib/ecs.py (#2556)
• FIX HdfsAtomicWriteDirPipe.close() when using snakebite and the file do not exist. (#2549)

Changed:

luigi:

• [ImgBot] optimizes images (#2555)

luigi.contrib:

• Remove s3 bucket validation prior to file upload (#2528)
• Refactor s3 copy into sub-methods (#2508)

2.7.9

Added

luigi.contrib:

• Added optional choice for hdfs clients (#2487)
• s3client check for deprecated host keyword and raise error with the details (#2493)
• Add a "capture_output" parameter to ExternalProgramTask (#2430)

Fixed

luigi:

• Fix exception when toml lib is not installed (#2506)
• Replace direct attribute accessing by using built-n function getattr (#2509)
• set upper bound of python-daemon (#2536)

luigi.contrib:

• Fix S3Client.copy return value consistency (#2488)
• Update MockTarget mode to accept r* or w* (#2519)