!!! FEATURE: High level Neos site import site:importAll

[bwaidelich]

Add commands site:importall, site:exportall and site:pruneall to import export and prune all sites of the given content repository. This will export events files and sites into a package (Resources/Private/Content) or any other specified target directory.

Further changes:

• Remove contentStreamId and workspaceName from exported events as those are overwritten on import anyways
• Remove commands cr:import, cr:export and cr:prune because those did not take the site nodes into account.

Todos

• More inline docs
• site:export command
• Add sites.json to export
• Add "check for empty content stream" processor (for better error handling)
• Extract workspace creation from events processor (only create if exists)
• Remove workspaceName and contentStreamId from events.jsonl
• Re-add lowlevel import/export service (to make review easier) I re-added the tests and test-tools for now, I would like to skip the services and cli commands for now (until the rest is "safe")
• Make LegacyMigrationService use high-level import
• \Neos\ContentRepository\LegacyNodeMigration\NodeDataToEventsProcessor (now \Neos\ContentRepository\LegacyNodeMigration\Processors\EventExportProcessor) has to work with workspaceName instead of contentStreamId -> solved

Resolves: #4448

[bwaidelich]

I decided to remove those low-level services for now.
It is quite easy to create them if needed (and more configurable then) but for Neos we'll need something more "highlevel" anyways

[bwaidelich]

I know, "context" is not the best pattern, but in this case it really makes sense IMO:
It's a common object that is passed through all processors allowing them a shared access to the (virtual) filesystem and to dispatch "events"

(needs doc comments)

[bwaidelich]

Just a plain (for now) collection object for processors that allows us to pass them around

[mficzel]

In #5306 we just moved cr:prune to the CrCommandController into the Neos.Neos->CrCommandController Package that gets deleted here.

[bwaidelich]

Looking great already, thanks for taking over. Just a few remarks

[mficzel]

Wonder about "Make LegacyMigrationService use high-level import" ... this would make the legacy migration depend on Neos.Neos ... technically there is no non Neos way to get an old Neos CR but it seems not that elegant anyways.

[mhsdesign]

Giving this +.5 already ;) will need to test that again with the Neos ui e2e tests (as they use the export as well) but for now it looks really good. We also have to adjust the flow setup to guide to these new commands :)

As written here #5307 (comment) im really questioning if migrateLegacyData should be a one step. We alreay require that people migrate to the Neos.Neos:Site homepage nodetype and their dimension config either way.

One more - good imo - step in between would just allow more flexibility and "come back later" to work on the migration.

[mhsdesign]

I'd be fine with that. As long as there is no cryptic exception when forgetting to execute a number of commands in a very specific order ;)

this would be it but could further be improved ... but idk ... :D Its just the flow command line exception handling printer :)

---

also im just testing this and it works so far, i guess its okay that the export overrides the directory specified but could also be dangerous?
maybe:

$directoryContainsFiles = (new \FilesystemIterator($path))->valid();
if ($directoryContainsFiles) {
    $this->outputLine(sprintf('Path %s is not empty.', $path));
    $this->quit(1);
}

[mhsdesign]

Okay i spend a lotta time testing this extensively now and fixing bugs and edgecases.

I tested doing a exportLegacyData with my local Neos 8 demo + domain and another site i use for Neos UI tests.
Then i imported that one again into Neos 9 testet the result via ui.
And to make it a full circle i exported that result again via exportAll and diffed the folders as we expect the same result - which it now finally is.

Following fixed bugs:

• imported sites are not active
• primary domain export was not correct
• having the same domain already was not a noop but crashed due to constraints
• fail early if --path is not a directory
• respect copyrightNotice of serialised assets during import

Now im at the state to say we can merge :D

[bwaidelich]

Thanks a lot for putting so much effort into this one, everybody!
Let's get this beast merged at some point. We can always tweak it later

[mhsdesign]

As per martin from slack

Fein

:D lets merge!

[kitsunet]

UI tests are now updated.

[mhsdesign]

Lol i just found an early draft in our git history eaf246c the export thing was now reworked also three times 😅 :D