Introduce tasks which produce intermediate results #126
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overhaul how tasks are defined in Colmena to provide the ability to distinguish tasks which produce intermediate results from those that do not.
Previously, we wrapped functions being invoked by a workflow engine in a function which handled the Colmena-specific things like unpacking results.
The new approach is simply a class-based strategy for wrapping. The logic which was in the wrapper function is now in the
__call__
method of the class, and the function being invoked is a attribute of the class.The class based approach gives more opportunities for creating subclasses, like the new
PythonGeneratorTask
and the legacyExecutableTask
class, which invoke the user-provided function differently.To do list: