Releases: kedro-org/kedro-plugins
Releases · kedro-org/kedro-plugins
kedro-datasets-6.0.0
Major features and improvements
- Supported passing
database
toibis.TableDataset
for load and save operations. - Added functionality to save pandas DataFrames directly to Snowflake, facilitating seamless
.csv
ingestion. - Added Python 3.9, 3.10 and 3.11 support for
snowflake.SnowflakeTableDataset
. - Enabled connection sharing between
ibis.FileDataset
andibis.TableDataset
instances, thereby allowing nodes to save data loaded by one to the other (as long as they share the same connection configuration). - Added the following new experimental datasets:
Type | Description | Location |
---|---|---|
databricks.ExternalTableDataset |
A dataset for accessing external tables in Databricks. | kedro_datasets_experimental.databricks |
safetensors.SafetensorsDataset |
A dataset for securely saving and loading files in the SafeTensors format. | kedro_datasets_experimental.safetensors |
Bug fixes and other changes
- Delayed backend connection for
pandas.GBQTableDataset
. In practice, this means that a dataset's connection details aren't used (or validated) until the dataset is accessed. On the plus side, the cost of connection isn't incurred regardless of when or whether the dataset is used. Furthermore, this makes the dataset object serializable (e.g. for use withParallelRunner
), because the unserializable client isn't part of it. - Removed the unused BigQuery client created in
pandas.GBQQueryDataset
. This makes the dataset object serializable (e.g. for use withParallelRunner
) by removing the unserializable object. - Implemented Snowflake's local testing framework for testing purposes.
- Improved the dependency management for Spark-based datasets by refactoring the Spark and Databricks utility functions used across the datasets.
- Added deprecation warning for
tracking.MetricsDataset
andtracking.JSONDataset
. - Moved
kedro-catalog
JSON schemas from Kedro core tokedro-datasets
.
Breaking Changes
- Demoted
video.VideoDataset
from core to experimental dataset. - Removed file handling capabilities from
ibis.TableDataset
. Useibis.FileDataset
to load and save files with an Ibis backend instead.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
kedro-telemetry-0.6.2
- Removed support for Python 3.8
- Added support for Python 3.13
Thanks for supporting contributions
kedro-docker-0.6.2
Major features and improvements
- Added support for Python 3.13
- Removed support for python 3.8
- Switch dependency management to
uv
.
Thanks for supporting contributions
kedro-airflow-0.9.2
- Removed support for Python 3.8
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
kedro-datasets-5.1.0
Major features and improvements
- Added the following new core datasets:
Type | Description | Location |
---|---|---|
ibis.FileDataset |
A dataset for loading and saving files using Ibis's backends. | kedro_datasets.ibis |
Bug fixes and other changes
- Changed Ibis datasets to connect to an in-memory DuckDB database if connection configuration is not provided.
kedro-datasets-5.0.0
Major features and improvements
- Removed support for Python 3.9
- Added the following new experimental datasets:
Type | Description | Location |
---|---|---|
pytorch.PyTorchDataset |
A dataset for securely saving and loading PyTorch models | kedro_datasets_experimental.pytorch |
prophet.ProphetModelDataset |
A dataset for Meta's Prophet model for time series forecasting | kedro_datasets_experimental.prophet |
- Added the following new core datasets:
Type | Description | Location |
---|---|---|
plotly.HTMLDataset |
A dataset for saving a plotly figure as HTML |
kedro_datasets.plotly |
Bug fixes and other changes
- Refactored all datasets to set
fs_args
defaults in the same way asload_args
andsave_args
and not have hardcoded values in the save methods. - Fixed bug related to loading/saving models from/to remote storage using
TensorFlowModelDataset
. - Fixed deprecated load and save approaches of GBQTableDataset and GBQQueryDataset by invoking save and load directly over
pandas-gbq
lib - Fixed incorrect
pandas
optional dependency
Breaking Changes
- Exposed
load
andsave
publicly for each dataset. This requires Kedro version 0.19.7 or higher. - Replaced the
geopandas.GeoJSONDataset
withgeopandas.GenericDataset
to support parquet and feather file formats.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
kedro-telemetry-0.6.1
- Changed Kedro CLI loading method to improve loading times.
- Changed logging level from error to debug for most logging messages.
- Set default value for the
identity
parameter, to prevent errors due to it being empty.
kedro-docker-0.6.1
- Unpinned pip version requirement
kedro-telemetry-0.6.0
- Moved to an opt-out model for telemetry, enabling it by default without requiring prior consent.
- Added
DO_NOT_TRACK
andKEDRO_DISABLE_TELEMETRY
environment variables to skip telemetry. - Reduced the number of events sent to Heap by eliminating duplicate events, improving efficiency.
- Enhanced telemetry functionality to support Kedro commands executed outside of Kedro project folders.
kedro-datasets-4.1.0
Major features and improvements
- Improved
partitions.PartitionedDataset
representation when printing.
Bug fixes and other changes
- Updated
ibis.TableDataset
to make sure credentials are not printed in interactive environment.