-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Codegen dataclassess #79
Codegen dataclassess #79
Conversation
4070a21
to
2973c17
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also need to emit dataclasses for the nested structs. We'll need some code to visit/iterate over all nested structs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend separating struct name computation from python type computation. We need functions/classes doing the following:
- Walking
Structs
to retrieve nestedStructs
- Assigning names to
Structs
- Computing types of fields given names of all the
Structs
involved - Generating code for data classes
If possible it would be great to minimise dependencies between them.
For example I would start by writing a generator recursively walking structs, something like this:
StructLink = typing.Tuple[Struct, Field, Struct]
def nested_structs(root: Struct) -> typing.Generator[StructLink, None, None]:
for field in root.fields:
type = field.type
match type:
case Array():
yield root, field, type.element_type
yield from nested_structs(type.element_type)
case TaggedUnion():
for struct in type.cases.values():
yield root, field, struct
yield from nested_structs(struct)
case Struct():
yield root, field, type
yield from nested_structs(type)
Such generator would then be a base for the algorithm assigning names to structs and algorithm generating all the dataclasses we want. For example:
def compute_struct_names(root: Struct, name: str) -> Mapping[Struct, str]:
names = {
root: name,
}
for parent, field, nested in nested_structs(root):
type = field.type
match field.type:
case Struct():
names[nested] = f"{names[parent]}{field.name.captialize()}"
case Array():
...
case TaggedUnion:
....
return names
# dedicated class to make it easier to pass all the necessary state around
class StructGenerator:
def __init__(self, root: Struct, name: str):
self.__struct_to_name = compute_struct_names(root, name)
def __get_python_type_for(self, struct: Struct, field: Field) -> str:
type = field.type
match type:
case Struct():
return self. __struct_to_name[type]
case Array():
return f"typing.List[{self.__struct_to_name[type.element_type]}]"
case TaggedUnion():
...
case _:
return types.python_type_for(type)
def __generate_dataclass(self, struct: Struct) -> str:
name = self.__struct_to_name[struct]
class_def = ...
return dedent(class_def)
def generate(self):
return [
self.__generate_dataclass(nested)
for _, _, nested in reversed(nested_structs(self.__root))
] + [self.__generate_dataclass(self.__root)]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests seem to be failing.
b6716db
to
60331c0
Compare
[jdwp] make jdwp schema immutable and hashable [jdwp] make struct serialization synchronous Codegen dataclassess (facebookexperimental#79) * Wrote test to validate primitive type mapping * Added test for type alias definition * Built dataclasses code generator * Built test for dataclassess code generator * Changed to unittest * Updated struct dataclass function to support all types * feat: built dataclass generator * Added copyright * Added nested struct support * Added test for nested struct * Feat: update functions and put it in a class. * chore: update tests * Casted types to pass pyre check * Updated types * chore: Refactor typing.cast code * Added copyright * refactor: update tests to reflect new schema changes * fix: Fix pyre typing errors [jdwp] do not emit array length and union tag fields [jdwp] map union tags and array lengths to int [jdwp] fix type for union fields [jdwp] fix names of fields containing spaces [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library implement async abstract methods, add TCP conn update files update conn and streams add buck deps
[jdwp] make jdwp schema immutable and hashable [jdwp] make struct serialization synchronous Codegen dataclassess (facebookexperimental#79) * Wrote test to validate primitive type mapping * Added test for type alias definition * Built dataclasses code generator * Built test for dataclassess code generator * Changed to unittest * Updated struct dataclass function to support all types * feat: built dataclass generator * Added copyright * Added nested struct support * Added test for nested struct * Feat: update functions and put it in a class. * chore: update tests * Casted types to pass pyre check * Updated types * chore: Refactor typing.cast code * Added copyright * refactor: update tests to reflect new schema changes * fix: Fix pyre typing errors [jdwp] do not emit array length and union tag fields [jdwp] map union tags and array lengths to int [jdwp] fix type for union fields [jdwp] fix names of fields containing spaces [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library implement async abstract methods, add TCP conn update files update conn and streams add buck deps add jvm connection and implement jvm streams
Codegen dataclassess (facebookexperimental#79) * Wrote test to validate primitive type mapping * Added test for type alias definition * Built dataclasses code generator * Built test for dataclassess code generator * Changed to unittest * Updated struct dataclass function to support all types * feat: built dataclass generator * Added copyright * Added nested struct support * Added test for nested struct * Feat: update functions and put it in a class. * chore: update tests * Casted types to pass pyre check * Updated types * chore: Refactor typing.cast code * Added copyright * refactor: update tests to reflect new schema changes * fix: Fix pyre typing errors [jdwp] do not emit array length and union tag fields [jdwp] map union tags and array lengths to int [jdwp] fix type for union fields [jdwp] fix names of fields containing spaces [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library implement async abstract methods, add TCP conn update files update conn and streams add buck deps add jvm connection and implement jvm streams add a base class for jdwp structs [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library fix fail build
Codegen dataclassess (facebookexperimental#79) * Wrote test to validate primitive type mapping * Added test for type alias definition * Built dataclasses code generator * Built test for dataclassess code generator * Changed to unittest * Updated struct dataclass function to support all types * feat: built dataclass generator * Added copyright * Added nested struct support * Added test for nested struct * Feat: update functions and put it in a class. * chore: update tests * Casted types to pass pyre check * Updated types * chore: Refactor typing.cast code * Added copyright * refactor: update tests to reflect new schema changes * fix: Fix pyre typing errors [jdwp] do not emit array length and union tag fields [jdwp] map union tags and array lengths to int [jdwp] fix type for union fields [jdwp] fix names of fields containing spaces [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library implement async abstract methods, add TCP conn update files update conn and streams add buck deps add jvm connection and implement jvm streams add a base class for jdwp structs [jdwp] generate projects.jdwp.runtime.structs [jdwp] add tests importing components of the runtime library fix fail build [jdwp] fix type checker fails
No description provided.