Splitting output in to separate files.

[shadowbane1000]

Would it be much work to have an option to have the compiler write it's output in to a separate file per class?

Before you read the stuff below, please be aware that I might have overlooked an error in the generated flatbuffer files that is causing this issue... But I don't think so.

I have a complex auto-converted collection of flat buffer tables/structs. If I have the converter write them all in to a single file, flatc.exe and FlatSharp can both generate C# source form them. However, due to a circular dependency in the files if I write the flat buffer data in to separate files per table/struct, both flatc.exe and FlatSharp cannot load the files. The ability to understand a circular dependency breaks on an include. The main reason I want to break them up is so that the C# is broken up so that developers don't have to deal with 50k lines of c# all in one file to try to understand things.

Alternately, making FlatSharp handle circular includes would work as well. In my imagination, this would require resolving all includes before trying to resolve types. As near as I can tell, this is not the way things work, and as FlatSharp has some kind of dependency on flatc.exe, I don't know that it will ever work without fixing flatc.exe.

[jamescourtney]

Hi @shadowbane1000 -- I've not been good at checking on the discussions section recently. I apologize for missing this for so long.

Can you raise an issue in google/flatbuffers about this? FlatSharp invokes flatc to get the itermediate representation of an FBS schema, which is functionally a FBS file translated into a FlatBuffer definition. FlatSharp then uses its reflection-only mode to parse that intermediate representation into an object model, which is then the starting point for the compiler. So, you can think of flatc as the "FBS parser" for FlatSharp, which is nice since flatc has a bunch of esoteric rules about type name resolution that are easy to break. And I have broken them :) So this is a big win for compatibility.

It's interesting that you suggest breaking classes up per-file. It's something I could maybe see exposing as an option. However, you should know that the upcoming FlatSharp v7 is actually going in the other direction, where the compiler will accept a big list of FBS files and produce one giant FlatSharp.generated.cs file as the output. The reason for this is that FlatSharp generates lots of duplicate code today if you reuse types extensively. Here's a small excerpt from the upcoming release notes for v7:

---

Previous versions of FlatSharp generated separate code for each root type. That is, imagine this schema:

// A big table
table Common
{
    A : string;
    B : string;
    ...
    Z : string;
}

table Outer1 (fs_serializer) { C : Common }
table Outer2 (fs_serializer) { C : Common }

In this case, FlatSharp 6 and below would generate 2 full serializers and parsers for Common, since it is used by both Outer1 and Outer2. This led to cases where commonly-used large tables would have more than one serializer implementation generated. Such code duplication leads to poor cache performance, poor utilization of the branch predictor, and an explosion of code when the type in question is used in multiple places, such as gRPC definitions.

FlatSharp 7 does the expected thing and generates only one serializer/parser for each type. As part of this change, FlatSharp 7 emits all serializer types, which comes with the happy accident of allowing you to specify it at runtime:

ISerializer<T> lazySerializer = Outer1.Serializer.WithSettings(settings => settings.UseLazyDeserialization());

---

Were you thinking that just the data definitions would get their own file, and the generated serializers and whatnot could get stuffed into a big "DontTouchThis.cs" style file? I'd like to try and accommodate you here if possible, but I need to understand a bit more about what you're trying to accomplish.

Thanks!