Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AvroSchemaGenerator: logicalType(s) never set for non-date classes #535

Open
credmond opened this issue Dec 11, 2024 · 6 comments
Open

AvroSchemaGenerator: logicalType(s) never set for non-date classes #535

credmond opened this issue Dec 11, 2024 · 6 comments
Labels
avro pr-needed Feature request for which PR likely needed (no active development but idea is workable)

Comments

@credmond
Copy link

logicalTypes doesn't appear to be set for anything other than dates.

    public static void main(String[] args) throws Exception {
        // Create AvroMapper and AvroSchemaGenerator
        AvroMapper avroMapper = AvroMapper.builder().addModule(new AvroJavaTimeModule()).build();

        AvroSchemaGenerator schemaGen = new AvroSchemaGenerator();
        schemaGen.enableLogicalTypes(); // Enable logical types

        // Configure AvroMapper with the generator
        avroMapper.acceptJsonFormatVisitor(MyClass.class, schemaGen);

        // Get the generated schema
        Schema schema = schemaGen.getGeneratedSchema().getAvroSchema();

        // Print the schema
        System.out.println(schema.toString(true));
    }

Output:

{
  "type" : "record",
  "name" : "MyClass",
  "namespace" : "com.fasterxml.jackson.dataformat.avro.jsr310",
  "fields" : [ {
    "name" : "someBigDecimal",
    "type" : [ "null", {
      "type" : "string",
      "java-class" : "java.math.BigDecimal"
    } ]
  }, {
    "name" : "someDouble",
    "type" : {
      "type" : "double",
      "java-class" : "java.lang.Double"
    }
  }, {
    "name" : "someDoubleObject",
    "type" : [ "null", {
      "type" : "double",
      "java-class" : "java.lang.Double"
    } ]
  }, {
    "name" : "timestamp",
    "type" : [ "null", {
      "type" : "long",
      "logicalType" : "timestamp-millis"
    } ]
  } ]
}

MyClass:

public class MyClass {
    private Instant timestamp;
    private double someDouble;
    private Double someDoubleObject;
    private BigDecimal someBigDecimal;

    // Setters / getters
}

@cowtowncoder
Copy link
Member

This is probably the state of things: logical type support was added a while ago via #283 and that's probably the extent.

Which other types would make sense?

As usual, PRs woud be welcome -- while I likely don't have time to work on this myself, I try my best to help anyone who wants to contribute.

@cowtowncoder cowtowncoder added the pr-needed Feature request for which PR likely needed (no active development but idea is workable) label Dec 12, 2024
@credmond credmond changed the title AvroSchemaGenerator: localTypes never set for non-date classes AvroSchemaGenerator: logicalType(s) never set for non-date classes Dec 12, 2024
@credmond
Copy link
Author

Hi -- actually I thought maybe it was a bug because there's even older closed issues/conversations about this, around 2018. I couldn't quite make sense of it but it seemed like it had been added in the past.

#133

Ends with "I think this is probably obsoleted by later work, closing.".

The other types that would make sense are "decimal" and "uuid" (there may be more as per spec, but I don't think so).

@cowtowncoder
Copy link
Member

@credmond My recollection (which may be incomplete for sure) is that date/time types were added but not much else. Adding more types should be straight-forward by someone digging in code (with guidance from earlier work, mainly #283).
I really think it's just omission, original contributor just needed certain types.
And I think Avro spec may have been updated too (although granted "decimal" and "uuid" were probably already mentioned).

Thank you for reporting omission in any case.

@cowtowncoder
Copy link
Member

cowtowncoder commented Dec 13, 2024

Ok yes, looking bit deeper, #133 would have added support for BigDecimal and UUID. But #283 did not, so there is a gap (latter is a subset of types former would have added). What it would have added (that seems useful) is @AvroType annotation, as well.

To add support, one place to look is where tests are, under com.fasterxml.jackson.dataformat.avro.schema.

@cowtowncoder
Copy link
Member

@cowtowncoder
Copy link
Member

Sort of related: #536 will add support for java.util.UUID wrt Avro Logical Types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
avro pr-needed Feature request for which PR likely needed (no active development but idea is workable)
Projects
None yet
Development

No branches or pull requests

2 participants