Status of this repo

[tekjar]

Hi. I'm on a similar path to implement kafka like distributed commit log. Is this project active?

[marceloboeira]

@tekjar I didn't have much time to work on the project lately. However, it would definitely be better to join efforts rather than starting it all over again. There is a lot of documentation here and I'm open to help and work on it given the time!

One of the biggest needs before moving to other steps is #4, which I'm stuck at the moment (#4 - Restoring the state from the disk log/index files).

The issue itself contains a lot of info on what has to be done, I'm a bit stuck on "how" with rust.

here is a WIP branch/commit. But feel free to work on it.
1fd9baf

After that, #16 is something useful, to sort of have a "release candidate" storage system and then move to the next steps: #5, #6, #8 and finally look into #12

[tekjar]

Sure I'll go through the details. Would you be interested in making this compatible with kafka clients? I'm not sure of the complexities but the ablity to replace just brokers without touching clients would be a big win

[marceloboeira]

I think the only thing I don't like about Kafka is that the server registers/keeps track of the cursor... I'm more inclined to believe that clients should keep track of where they are in the stream and don't need to be registered.

However, I believe such discussions are quite far away and we can easily implement both models once the base storage and replication layers are "done".

[tekjar]

Ok. I'll go through the details of the storage layer first

[tekjar]

Hi. I started my work. I'm doing some refactors to get myself familiar first.

Is there a better way to communicate other than Github issues? Something like discord? Not a big deal but helps me understand and fix things faster

[marceloboeira]

We can start one, feel free to refactor what feels like not working. I'm far from being a rust expert.

Good point for the communication, I've just created a discord server for voik, you can join with https://discord.gg/TvMjtr

I'll close this issue for the time being, we can move the discussion to discord.

[andoriyu]

@marceloboeira looks like discord link is dead?