Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-18023: Enforcing Explicit Naming for Kafka Streams Internal Topics #18233

Open
wants to merge 4 commits into
base: trunk
Choose a base branch
from

Conversation

sebastienviale
Copy link
Contributor

@sebastienviale sebastienviale commented Dec 17, 2024

Pull request to implement KIP-1111, aims to add a configuration that prevents a Kafka Streams application from starting if any of its internal topics have auto-generated names, thereby enforcing explicit naming for all internal topics and enhancing the stability of the application’s topology.

  • Repartition Topics:

All repartition topics are created in the KStreamImpl.createRepartitionedSource(...) static method. This method either receives a name explicitly provided by the user or null and then builds the final repartition topic name.

  • Changelog Topics and State Store Names:

There are several scenarios where these are created:
In the MaterializedInternal constructor.
During KStream/KStream joins.
During KStream/KTable joins with grace periods.
With key-value buffers are used in suppressions.

@github-actions github-actions bot added triage PRs from the community streams labels Dec 17, 2024
@sebastienviale
Copy link
Contributor Author

sebastienviale commented Dec 17, 2024

There are some missing asserts in unit tests. I will add them, especially once I am confident that the implementation is on the right track.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
streams triage PRs from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant