Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: Enable Kafka integration #1021

Merged
merged 11 commits into from
Dec 20, 2024
Merged

Feature: Enable Kafka integration #1021

merged 11 commits into from
Dec 20, 2024
+400 −1

Conversation

parmesant
Copy link
Contributor

@parmesant parmesant commented Dec 5, 2024 •
edited
Loading

Fixes #936

Description

For this to work, these env vars are mandatory-

  • P_KAFKA_TOPIC
  • P_KAFKA_HOST
  • P_KAFKA_GROUP

Enables kafka-integration for parseable

Note
libz support has been turned off for now
The other option was to force users to use libz which we decided against

This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

@parmesant parmesant force-pushed the kafka-integration branch 4 times, most recently from f5a6d5b to 4347396 Compare December 10, 2024 05:41
de-sh
de-sh suggested changes Dec 13, 2024
View reviewed changes
src/cli.rs Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
src/kafka.rs Outdated Show resolved Hide resolved
@coveralls
Copy link

coveralls commented Dec 15, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 12412610022

Details

  • 0 of 153 (0.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.1%) to 11.441%
Changes Missing Coverage Covered Lines Changed/Added Lines %
src/cli.rs 0 49 0.0%
src/kafka.rs 0 104 0.0%
Totals Coverage Status
Change from base Build 12411844504: -0.1%
Covered Lines: 2096
Relevant Lines: 18320
💛 - Coveralls

@nitisht nitisht merged commit 4d3437c into parseablehq:main Dec 20, 2024
6 checks passed
@de-sh de-sh mentioned this pull request Dec 20, 2024
Merged
3 tasks
nitisht pushed a commit that referenced this pull request Dec 21, 2024
@de-sh
fix: incompatibility from merge (#1046)
503eaa0 
Fixes type incompatibility that stems from merging #1021 after #981
@nitisht nitisht mentioned this pull request Dec 21, 2024
Draft
7 tasks
parmesant added a commit to parmesant/parseable that referenced this pull request Dec 24, 2024
@parmesant @nitisht @de-sh
Feature: Enable Kafka integration (parseablehq#1021)
1b5d433 
Fixes parseablehq#936

---------

Signed-off-by: parmesant <[email protected]>
Co-authored-by: Nitish Tiwari <[email protected]>
Co-authored-by: Devdutt Shenoi <[email protected]>
parmesant pushed a commit to parmesant/parseable that referenced this pull request Dec 24, 2024
@de-sh @parmesant
fix: incompatibility from merge (parseablehq#1046)
f625975 
Fixes type incompatibility that stems from merging parseablehq#1021 after parseablehq#981
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@de-sh de-sh de-sh approved these changes

@nikhilsinhaparseable nikhilsinhaparseable Awaiting requested review from nikhilsinhaparseable

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Feat: Add native Kafka integration for Parseable server
4 participants
@parmesant @coveralls @de-sh @nitisht