You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we have a misalignment of responsibilities between core and datasets domains, when it comes to server basic dataset use cases, such as:
create dataset
crate dataset via snapshot
rename dataset
delete dataset
Right now the implementation of use cases is fully concentrated in infra/core crate, and the datasets domain handlers, such as dataset entry maintainance, dependency graph updates, and future planned actions like #978, are integrated via Outbox events of immediate delivery type.
This feels misaligned, as basically we are talking about the very same domain, and it should not be that difficult to synchronously update both the storage and the datasets domain records.
These key datasetuse cases already abstract away from storage system via DatasetRepositoryWriter trait, and currently DatasetRepositoryLocalFs and DatasetRepositoryS3 implement it directly.
Proposal: provide an alternative implementation for DatasetRepositoryWriter in datasets domain, which would:
make CRUD operations on dataset entries directly, instead of expecting immediate events
make pre-validations against the database vision first (storage versions should be simplified, i.e. naming collision detection)
update dependency graph directly
set initial HEAD reference in database when creating dataset in the same transaction as creating a dataset entry
invoke storage repositories to do their part within the same transaction
The text was updated successfully, but these errors were encountered:
Blocked with #979
Currently we have a misalignment of responsibilities between
core
anddatasets
domains, when it comes to server basic dataset use cases, such as:Right now the implementation of use cases is fully concentrated in
infra/core
crate, and thedatasets
domain handlers, such as dataset entry maintainance, dependency graph updates, and future planned actions like #978, are integrated viaOutbox
events of immediate delivery type.This feels misaligned, as basically we are talking about the very same domain, and it should not be that difficult to synchronously update both the storage and the
datasets
domain records.These key datasetuse cases already abstract away from storage system via
DatasetRepositoryWriter
trait, and currentlyDatasetRepositoryLocalFs
andDatasetRepositoryS3
implement it directly.Proposal: provide an alternative implementation for
DatasetRepositoryWriter
indatasets
domain, which would:The text was updated successfully, but these errors were encountered: