5.7.0

[jamescourtney]

FlatSharp 5.7.0 adds the most commonly requested feature: Value-type structs. Value type structs can be used from both C# and from .fbs files, but work best from FBS files:

struct ValueStruct (fs_valueStruct)
{
       A : int;
       B : ulong;
       Vector : [ ubyte : 32 ];
}

FlatSharp value structs...

• Can only contain other value types (value structs and scalars). You cannot embed reference structs inside value structs. However, value structs can be embedded in reference structs.
• Support struct vectors. Due to value type semantics, the implementation is a little different than struct vectors for reference structs. The struct vectors sample has more detail.
• Are generally serialized and deserialized with a memcopy operation. Here's a typical value struct serializer:

Span<byte> sizedSpan = span.Slice(offset, 41);
if (BitConverter.IsLittleEndian)
{
    var tempSpan = MemoryMarshal.Cast<byte, Vec3Value>(sizedSpan);

    // Assign the struct into the underlying buffer. No need to write field-by-field since the struct's
    // memory layout is the same as the flatbuffer layout!
    tempSpan[0] = value;
}
else
{
    WriteInlineValueOf(spanWriter, sizedSpan, value.temp, 0);
    ...
}

• Are never incrementally deserialized (it's not possible to only deserialize valueStruct.Property1 -- all of valueStruct is read and then you can access property1).

Full samples are available here.

5.7.0 has a few other assorted changes as well:

• Better parallelism if using runtime serializer generation
• Better error messaging when using the FlatSharp compiler (the C# errors are written to the output file)
• The testing infrastructure has been switched to xUnit.

Some types are now marked as obsolete. These are being considered for removal or are scheduled for removal in FlatSharp version 6.0.0, which is an upcoming release without a firm date or feature set.

• The FlatSharp.Unsafe package is scheduled for removal in version 6.0.0.
• The IFlatBufferAddressableStruct interface is being considered for removal in version 6.0.0, because its primary purpose was enabling marshaling from reference -> value types, and this is now supported natively with value structs. If you want to keep using IFlatBufferAddressableStruct, even with the value structs feature, please raise an issue in this repository.
• Shared String Readers (but not writers) are being considered for removal in version 6.0.0. Readers generally do not offer a performance benefit and are slower than simply re-reading the string.

---

This discussion was created from the release 5.7.0.

[jamescourtney]

@TYoungSL, @Astn, and @tdmowrer -- this may be relevant to you.

[Astn]

I think that is ok

[jamescourtney]

I've been thinking about WriteThrough a bunch over the last few days, and wanted to take your pulse on a question I'm having. The long-and-short of the problem is that reference types can do WriteThrough and other neat things because of polymorphism / inheritance. We don't get those benefits with value types.

There is no good way to do WriteThrough with the current value type implementation because everything is done with fields and not properties.

Since value structs are copied on assignment, there would be no way to keep all the "copies" in sync with something like WriteThrough, unless they all pointed at the same backing buffer and re-read the value each time on access. So they would look something like this:

public struct WriteThroughStruct
{
        private readonly Memory<byte> buffer;
        
        public int Foo
        {
               get => BinaryPrimitives.ReadInt32LittleEndian(this.buffer.Span.Slice(0, 4));
               set => BinaryPrimitives.WriteInt32LittleEndian(this.buffer.Span.Slice(0, 4), value);
        }
}

This looks nice, but it is slow. Accessing a property based on a Memory<byte> buffer is about 10x slower than accessing a field on a struct (what the current implementation does). I actually wasn't even able to do better than this using unsafe code (turns out, pinning a memory buffer in place to get the raw pointer is more expensive than the bounds check!).

So in this world, I need to pick one of these doors for flatsharp:

• Structs are defined as they are today, with LayoutKind.Explicit. All members are fields.

  • [+] It acts like a struct. It's super fast to access and pretty fast to serialize/parse. No mandatory heap allocations ever.
  • [-] For large structs, copying it around will be slow (But you should consider a reference struct in this case, so I hesitate to call this a con since the library provides you with an alternative).

• Structs become "buffer-backed", much like they are in the canonical FlatBuffer library and the example I gave above.

  • [+] Value structs can be fancy and approach some functionality of reference structs, such as write through.
  • [+] Copies of large value structs are cheap, since you're really just moving a pointer to the real buffer on each copy.
  • [--] You give up a lot of performance, possibly negating the point of using a value struct in the first place. A virtual member on a class is about twice as fast (!) a buffer-backed struct method like I showed in the example.
  • [--] If you want to create a new WriteThrough struct (possibly to store in a buffer, etc), you are also required to allocate a new backing byte array for it to store its data in: new WriteThroughStruct(new byte[WriteThroughStruct.Size]), which -- again -- may negate the point.

So, it seems to me that buffer-backed value structs are simply worse versions of reference structs. I think I'd rather bet on the GC becoming more efficient and concurrent than try and shoehorn in some worst-of-both-worlds approach.

But I'm curious to see what you and others (@TYoungSL) might think. Am I missing something?

[Astn]

I really like the LayoutKind.Explict approach.

I think it enables a lot of use cases by knowing the memory layout. Like having the option to write to a vector pointer of that struct. Or simply write the backing buffer out to the network, or disk, or whatever.

I know I have use cases where this would be an advantage. I would be able to completely skip the de-serialization and re-serialization process and still send slices of vectors out the network just by having access to a span of the data I want.

[jamescourtney]

Thanks (as always!) for your thoughtful response. I really appreciate having someone that's as engaged as you to bounce things off of :)

One thing I've been kicking around for writethrough is an API like this on a deserialized table:

public interface IFlatBufferDeserializedTable : IFlatBufferDeserializedObject
{
    // The reason for adding the generic parameter here is to allow the JIT to exclude entire branches for string comparison.
    (int offset, int length)? OffsetOf<TStruct>(string propertyName, int? index = null) where TStruct : struct;

    void Save<TStruct>(TStruct value, string propertyName, int? index = null); where TStruct : struct;
}

Usage would look roughly like:

if (table is IFlatBufferDeserializedTable dTable)
{
    (int offset, int length)? point = table.OffsetOf<Vec3>("points", 3);
    
}

This would enable querying the offset and size of any property in the table (for vectors, you'd need to specify the index). This accomplishes the same goals as IFlatBufferAddressableStruct, but this is done from the containing table instead. It seems pretty similar to what you're describing about slicing the underlying array (though you could get the same effect from just using Lazy deserialization -- which is just slicing the underlying array on-demand). Just be careful about endianness if you're doing the casting yourself!

For the record, I re-ran your voxel benchmarks on value structs. I don't have the results with me on my laptop, but the value structs were in some cases even faster than the write-through reference structs, which is a great result. I'll post the benchmarks later once I get back to my desktop machine.

[Astn]

Yes, very cool. And glad I could be any help.

I think the more information you able to get access to about what is in contained in the buffer without the costs of parsing the buffer will eventually enable shorter code path optimizations.

Like I love being able to know how big a vector is prior to de-serializing it as long as that is a cheep operation.

You know... It'd be fun to chat a bit. Are you on discord? Friend me (Lord Rent#6681) on there.

[jamescourtney]

Just published 5.7.1 on Nuget. It's a small update on top of 5.7.0. I found some interesting/horrible things about MemoryMarshal.Cast. The performance of that method is.. unpredictable. So, the main things in 5.7.1 are:

• @TYoungSL's change to my crappy unsafe code. Thanks again :)
• Added a bunch more test around MemoryMarshal.Cast
• Added compiler errors if you try to do fs_writeThrough on a value struct, or unsafeVector on a reference struct.
• Addition of one more metadata: https://github.com/jamescourtney/FlatSharp/wiki/FBS-Annotations-(FlatSharp-5.X)#fs_memorymarshal

Regarding MemoryMarshal.Cast, here's a quick and dirty benchmark I ran earlier today. The benchmark was serializing a table with 20,000 of these structs in a giant vector, and the numbers are in microseconds. So none of these are exactly slow. However, the behavior is maddeningly inconsistent.

NetCore2.1
Size	Serialize_MemMarshal	Serialize_FieldByField	Parse_MemMarshal	Parse_FieldByField
1 int32	34	24	262	55
2 int32	94	25	262	152
3 int32	43	26	269	105
4 int32	52	1065*	267	126
5 int32	47	59	274	249
NetCore3.1
1 int32	30	13	256	47
2 int32	93	18	259	151
3 int32	43	19	264	110
4 int32	56	33	272	126
5 int32	44	64	269	197
Net50
1 int32	34	13	75	50
2 int32	93	17	76	147
3 int32	86	19	88	109
4 int32	36	32	76	121
5 int32	47	60	88	196

.Net 5.0 behaves in a (mostly) sane way, though it still loses its mind a bit with 2 and 3 item structs. The table above is only for int32 items, but I observed the same "spike" at 2 and 3 item structs for byte, ushort, and ulong, which means the spike is based on the number of items, not the size of the struct. It's really weird.

.Net Core 2.1 seems reasonably consistent, but goes extra crazy on a 4 item struct when doing field-by-field serialization. The result is repeatable on my system, but only for int32 in this case.

In any case, this is now all configurable. Have fun. The default heuristic I ended up using if you don't pick a value for memmarshal is:

• If the struct has only scalar children and is < 4 items (independent of size), do field by field
• Otherwise, use MemoryMarshal

[jamescourtney]

Some observations about this:

    public void Serialize_Marshal(Span<byte> target, int offset, Vec3 item)
    {
        Span<byte> sized = target.Slice(offset, 12);
        MemoryMarshal.Cast<byte, Vec3>(sized)[0] = item;
    }

    [StructLayout(LayoutKind.Explicit, Size = 12)]
    public struct Vec3
    {
        [FieldOffset(0)] public float X;
        [FieldOffset(4)] public float Y;
        [FieldOffset(8)] public float Z;
    }

The JIT ASM for Serialize_Marshal on .NET 5 looks like:

C.Serialize_Marshal(System.Span`1<Byte>, Int32, Vec3)
    L0000: sub rsp, 0x28
    L0004: mov eax, r8d
    L0007: add rax, 0xc
    L000b: mov ecx, [rdx+8]
    L000e: cmp rax, rcx
    L0011: ja short L003d
    L0013: mov rax, [rdx]
    L0016: movsxd rdx, r8d
    L0019: add rax, rdx
    L001c: mov edx, 1
    L0021: cmp edx, 0
    L0024: jbe short L0043
    L0026: mov rdx, [r9]
    L0029: mov [rax], rdx
    L002c: mov edx, [r9+8]
    L0030: mov [rax+8], edx
    L0033: add rsp, 0x28
    L0037: ret
    L0038: call 0x00007ffc7802a080
    L003d: call System.ThrowHelper.ThrowArgumentOutOfRangeException()
    L0042: int3
    L0043: call 0x00007ffc7802a5e0
    L0048: int3

The interesting thing about this is that it has a completely redundant jump:

    L001c: mov edx, 1
    L0021: cmp edx, 0
    L0024: jbe short L0043

In this case, edx will always be 1, so we already know the answer of comparing it to 0, which means we'll never take the branch.

[jamescourtney]

https://sharplab.io/#gist:393f718b8dff1501e3b5279410163780