1.2.1

This patch release addresses the following items:

1. Out-of-memory errors occurring when reading data with the reprocess command have been addressed via a fix in the underlying MarkLogic Java Client library.
2. The --embedder option for specifying an embedding model no longer lowercases the value provided to it.

1.2.0

This release provides the following enhancements:

1. Support for Retrieval-augmented generation, or RAG with MarkLogic via support for splitting text and adding embeddings during any import or copy operation. See the Import guide for more information on splitting text and adding embeddings.
2. When exporting rows, can now use any Optic data access function and not just op.fromView.
3. When exporting documents, Flux no longer requires any query options to be specified and defaults to exporting all documents that a user can read.
4. When exporting data to files, Flux will automatically create the given path if it does not exist, regardless of the target file type (prior to 1.2.0, this only happened for export commands).
5. The reprocess command has a new --thread-count option to simplify parallelizing the processing of data in MarkLogic.

To get started, download the marklogic-flux-1.2.0.zip file below and visit our Getting Started guide.

1.1.3

This patch release addresses a single issue - an unused transitive dependency (via Spark and Hadoop) on log4j 1.2.17 is no longer included in Flux. Flux did not make use of this dependency in any of its prior releases, and it can be safely removed from the ./lib folder in prior releases.

Note that while this unused log4j dependency has several open CVEs assigned to it, it is not impacted by the LogShell log4j vulnerability. Flux has never been impacted by this vulnerability, as it has used log4j 2.19.0 or higher since its 1.0.0 release.

This release is otherwise identical to the Flux 1.1.2 release, and in fact is equivalent to Flux 1.1.2 once the log4j 1.2.17 jar is removed from the Flux 1.1.2 ./lib folder.

1.1.2

This patch release addresses the following two issues:

1. Export commands that cannot depend on the requirement of a consistent snapshot when querying MarkLogic can now use the new --no-snapshot option so that Flux queries MarkLogic at multiple times for data. Please see the user guide for more information on consistent snapshots and when to use them when exporting data.
2. JSON Lines files can now be imported "as is" via the new --json-lines-raw option. Please see the user guide for more information on this fix for avoiding modifications to the JSON documents associated with each line in a JSON Lines file.

1.1.1

This patch release addresses an issue with logging when running Flux on Linux. It is otherwise equivalent to the 1.1.0 release.

1.1.0

This minor release provides the following enhancements and fixes:

1. The import-files, import-archive-files, export-files, and export-archive-files all support a new --streaming option that ensures files and documents from MarkLogic are never read into memory. This is intended to be used when importing or exporting large binary files that either Flux and/or MarkLogic cannot read into memory. Please see the user guide for more details - you can search on "streaming" to find the relevant sections.
2. Commands that export data to files now default to a save mode of append instead of overwrite.
3. All commands that import files now support filenames with spaces in them.

For users running Flux on Linux - please use the 1.1.1 release as this fixes a logging configuration issue that only occurs on Linux.

1.0.0

This is the first major release of the Progress MarkLogic Flux application for supporting data movement use cases with the MarkLogic data platform.

Please see the Flux user guide for information on how to use Flux.

To obtain the Flux application, download the marklogic-flux-1.0.0.zip file below and see the Getting Started guide.

If you are interested in using Flux with the Apache Spark spark-submit tool, download the marklogic-flux-1.0.0-all.jar and see the Spark Integration guide.

For any questions, enhancement requests, or bug reports, you can file an issue in this repository or contact Progress MarkLogic Support if you are a licensed customer. Progress MarkLogic provides technical support for this release to licensed customers under the terms outlined in the Support Handbook.