-
Notifications
You must be signed in to change notification settings - Fork 221
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make TextualFromNative output deterministic #254
base: master
Are you sure you want to change the base?
Make TextualFromNative output deterministic #254
Conversation
Hi and thanks! I'm worried about this having a perf impact. goavro has been very focused on perf. Go has a decent framework for running benchmarks. Would you please include a benchmark test with this PR as in this PR #223 |
Understood, will do as soon as I find available time |
On my branch
On master
@xmcqueen What did you had in mind, do you want me to add some really large map with 1M records and compare or? |
Another way to make it deterministic that would maybe be faster than sorting the keys would be to keep the order of the fields in the records in the schema. It would be consistent with what produce Avro library for Java and also with avro-tools. Here I think the function should should use For map, of course there is no definitive order. The Java implementation uses |
This is my take on resolving the #222.
I used a simple approach of extracting the keys and sorting them in a deterministic way.
Hopefully this is ok or can add some perspective to this issue