Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple Fixes for Distributed Deployments #786

Merged

Conversation

nikhilsinhaparseable
Copy link
Contributor

@nikhilsinhaparseable nikhilsinhaparseable commented Apr 29, 2024

  • to subtract the size of the arrow file removed from staging from staging size metrics
  • to return error in case global cache is not set - /GET cache returns true/false but does not even check if global cache was enabled
    change done is to return error if global cache is not set, if set return true/false
  • query fix for migration from standalone to distributed deployments - query was not working for the manifest listed dates but was working when from date was given a day before the manifest listed date
    change done is to add querier manifest.json also while checking for all manifests related to the time filter
  • fix for edge case when ingestor was down/unreachable at the timeout of retention activity and ingestor's stream.json did not get updated now when ingestion happens from ingestor, local to storage sync fails with error - Manifest found in snapshot but not in object-storage change done is to create the manifest but not update the snapshot because snapshot already has the manifest entry

- to subtract the size of the arrow file removed from staging
fix for GET /cache
- to return error in case global cache is not set
from standalone to distributed deployments
@nikhilsinhaparseable nikhilsinhaparseable changed the title fix for staging size metrics and GET /cache Multiple Fixes for Distributed Deployments Apr 30, 2024
from standalone to distributed
- querier should cleanup its snapshot upon retention
…f retention activity

and ingestor's stream.json did not get updated
now when ingestion happens from ingestor,
local to storage sync fails with error -
Manifest found in snapshot but not in object-storage
change done is to create the manifest but not update the snapshot
because snapshot already has the manifest entry
Copy link
Contributor

@Eshanatnight Eshanatnight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Good

@nitisht nitisht merged commit d4dcd0e into parseablehq:main May 1, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants