Add more versatile replication filters

[cedvan]

Hi all,

Need

I want replicate images from a namespace. But only need replicate tags "stable" to save my storage

On docker hub :

• 1.10.0-alpha
• 1.9.01
• 1.9.0
• 1.9.0-rc2
• 1.9.0-rc1
• 1.8.0

Need get on my harbor :

• 1.9.01
• 1.9.0
• 1.8.0

So ignore pre release version and alpha version to save storage.

Problem

Try configure my replication rule with existing filters (cf. https://github.com/goharbor/harbor/blob/master/docs/user_guide.md#resource-filter)

• *.*.* => but 1.9.0-rc1 match
• ?.?.? => but 1.9.01 not match
• {?.?.?,?.??.??} => Ok, but 1.10.0 soon available and so don't match

Problem : It is necessary to update the filter regularly so that my replication rule works in the future

Proposal solutions

1 - Add a pattern to match only number, like \d with regex
2 - Add more versatile replication filter like composer *, ~, ^, <, =, > (cf. https://getcomposer.org/doc/articles/versions.md#versions-and-constraints)
3 - Simply use regex as pattern to configure filter

[jcmartins]

+1

[xaleeks]

we need a common consistent regex support for all features, not just limited to replication. Adding this to next release planning

[dkulchinsky]

@xaleeks was wondering if there's any tentative timeline for this work (this is still tagged for 1.11.0)? I totally agree that there need to be a consistent regex support throughout the different features, our immediate use case is with immutability rules.

[HaveFun83]

as workaround we use this filter {?,??}.{?,??}.{?,??} for some kind of semver matching

[cedvan]

@HaveFun83 This solution not match versions "X" and "X.X"

[HaveFun83]

@HaveFun83 This solution not match versions "X" and "X.X"

sure "X" and "X.X" is not a semver annotation so far as i know.
https://semver.org/#spec-item-2

[dkulchinsky]

Hey @xaleeks, is there any update you can provide on this? would love to this get slotted for a future version 🙏🏼

[badsmoke]

this topic has also just come up in our company, it would be fabulous if this feature were to be implimented soon.

[veyselsahin]

We are waiting for this feature, too.

[badsmoke]

I have the feeling that this thread receives no attention :-(

[cedvan]

1.11.0 candidate, last version is 2.2.2...

[niklasweimann]

I would love to see this feature in the next release. It's super annoying without this feature

[exight]

This feature would be absolut awesome!

[UrsusLvovich]

My company would also love to have regex support in the tag filtering

[wy65701436]

ping @qnetter for awareness

[Aadeesh11]

I have applied for this project on LFX website. I have started learning GO, so can I get some pointers on how to get started with this project.

[WilfredAlmeida]

This issue seems like a great starting point for getting started with a contribution to Harbor. Landed here from the LFX Mentorship projects.

I'd like to get started with the issue regardless.

Can someone point out the file where the regex is written for the filters for me to take a look?

[chlins]

https://github.com/goharbor/harbor/blob/main/src/pkg/reg/filter/artifact.go

[Vad1mo]

I would opt for option 3.
Paired with a fast regex lib like re2, it should be possible to filter a big chunk of repos at once

[WilfredAlmeida]

I would opt for option 3. Paired with a fast regex lib like re2, it should be possible to filter a big chunk of repos at once

So will the function IsSemanticVersion() be similar to the other functions in pattern.go do? It'll have regex-matching semantic versioning numbers.

[Vad1mo]

We should also, make sure that the new feature is backwards-compatible.

I see two options.

1. migrate/convert existing rules into regex
2. add an option for the user to select a simple option or regex.
   ( I, personally, think this many options would make the understanding too complex)

I would prefer option 1)

[WilfredAlmeida]

migrate/convert existing rules into regex

Agreed. This can be done. There are 2 approaches to doing this

1. Migrate existing rules to regex, then add for this issue
2. Implement current issue filters in regex and then migrate existing rules

I'd prefer option 2 since it'll give us more clarity into regex.

[WilfredAlmeida]

I've drafted a proposal for this issue formulating multiple approaches to solve the issue, please review and share your thoughts.

goharbor/community#221