Distribution specific override

[2maz]

Add an overrides.d folder in package sets, and allow for OS-specific overrides (e.g. overrides.d/ubuntu/20.04)

[2maz]

@planthaber tried to follow your suggestions - see update

[planthaber]

When the package_set is removed form the manifest, the symlink is not deleted causing an error

[g-arjones]

You also have to deal with overrides files that may be removed from the package set. Managing this and user overrides in the same place won't be straightforward.

Why not load the files from where they are rather than create symlinks?

[2maz]

You also have to deal with overrides files that may be removed from the package set. Managing this and user overrides in the same place won't be straightforward.

Should now be handled by a cleanup of all dangling symlinks, or do I miss anything there?

Why not load the files from where they are rather than create symlinks?

IMO this gives more transparency (user can check overrides.d folder in the main configuration) on what overrides actually apply and in what order.
As a sidenote: loading from an overrides.d folder is currently limited to local package sets - not clear to me what the original motivation was to not implement it for package set in general.

[g-arjones]

or do I miss anything there?

Yes, overrides that are removed because a more specific one was added without deleting the default. The symlink will not be dangling but should be removed anyway.

user can check overrides.d folder in the main configuration) on what overrides actually apply and in what order

That's not recommended because one would miss entries on the overrides section of the source.yml file. To check for overrides autoproj show is still the best option

[2maz]

K, how about the updated version then - changed to loading from overrides.d folder from within package set.

Note that #335 should be applied first, otherwise the load order of the package sets might break the intended overrides.

[doudou]

The reason there is no overrides.d in the package sets was that it was in principle not needed. There is a way to override from the source.yml, and package sets are not a place where one needs to be dropping files to add new overrides "side-by-side" with the stuff that gets managed by the vcs.

Anyways

I completely understand the need, however I'm not too fond of the proposed mechanism.

The overrides that get added by this mechanism will be applied at the very end of everything. I.e. a package set that's "downstream" to e.g. rock-core would be overriden by stuff from rock-core. I think it will make for a very fragile mechanism.

I would personally have preferred to add a osdeps-like structure within source.yml, which would at least be consistent with the osdeps system, and would allow to apply the same patterns than within the osdeps (e.g. have rules that apply to both debian & ubuntu, group rules for different versions and then have a default ...)

[2maz]

The overrides that get added by this mechanism will be applied at the very end of everything. I.e. a package set that's "downstream" to e.g. rock-core would be overriden by stuff from rock-core. I think it will make for a very fragile mechanism.

If overrides are applied in import order, I do not see a problem?!

I would personally have preferred to add a osdeps-like structure within source.yml, which would at least be consistent with the osdeps system, and would allow to apply the same patterns than within the osdeps (e.g. have rules that apply to both debian & ubuntu, group rules for different versions and then have a default ...)

Thought about it, but I still think that having overrides in separate file/folder structure makes the handling easier

[doudou]

If overrides are applied in import order, I do not see a problem?!

There is no problem. My apologies, I got lazy and replied with the state of the code from a few days ago.

Thought about it, but I still think that having overrides in separate file/folder structure makes the handling easier

This is obviously a matter of taste ... I despise having a lot of small folders. Especially since one will probably have a very small number of folders with a single file per folder in most cases.

Note that by having the osdep-like mechanism one can do both. Just name your overrides file xx_ubuntu_2004.yml by convention and use the osdep-like structure in the file.

Now, adding that loading mechanism is quite a bit of work ... This (I don't have the f... time) is an argument I get ;-)

[doudou]

(I rewrote the PR description to reflect the current state of the code)

[g-arjones]

To be consistent with the osdeps system, I think you should also be allowing i.e:

overrides.d/ubuntu,debian/file.yml
overrides.d/ubuntu/18.04,18,10,19.04/file.yml