Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimise indexer active crawling process #3447

Open
t83714 opened this issue Feb 21, 2023 · 0 comments
Open

optimise indexer active crawling process #3447

t83714 opened this issue Feb 21, 2023 · 0 comments

Comments

@t83714
Copy link
Contributor

t83714 commented Feb 21, 2023

optimise indexer active crawling process

The indexer's active crawling process can be improved in the following areas:

  • The indexer should temporarily disable the indexer webhook after the active crawling process starts and re-enable the webhook once it completes.
  • The indexer should record the latest event number of the system, and if this number is higher than the event number that the indexer has processed up to, the indexer webhook lasteventid will be replaced with this number.
    • This makes sure indexer will not re-do the events that have been processed by the active crawling process.
@t83714 t83714 added this to the v2.2.3 milestone Feb 21, 2023
@t83714 t83714 modified the milestones: v2.2.3, v2.2.4 Apr 2, 2023
@t83714 t83714 modified the milestones: v2.2.4, v2.2.5 May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant