VYaml

VYaml is a pure C# YAML 1.2 implementation, which is extra fast, low memory footprint with focued on .NET and Unity.

• The parser is heavily influenced by yaml-rust, and libyaml, yaml-cpp.
• Serialization interface/implementation is heavily influenced by Utf8Json, MessagePack-CSharp, MemoryPack.

The reason VYaml is fast is it handles utf8 byte sequences directly with newface api set of C# (System.Buffers.*, etc). In parsing, scalar values are pooled and no allocation occurs until Scalar.ToString(). This works with very low memory footprint and low performance overhead, in environments such as Unity.

Compared with YamlDotNet (most popular yaml library in C#), basically 6x faster and about 1/50 heap allocations in some case.

Currentry supported fetures

• YAML Parser (Reader)
  • YAML 1.2 mostly supported
  • Support Unity serialized weird YAML format
    • https://forum.unity.com/threads/scene-files-invalid-yaml.355653/
    • YAML automatically generated by Unity may contain the symbol "stripped" in the document start line. This is against the YAML specification, but VYaml supports this format.
• YAML Emitter (Writer)
  • Write primitive types.
  • Write plain scalar, double-quoted scalar, literal scalar.
  • Write block style sequence, flow style sequence, and block mapping.
• Deserialize / Serialize
  • Convert between YAML and C# user-defined types.
  • Convert between YAML and primitive collection via dynamic .
  • Support interface-typed and abstract class-typed objects.
  • Support anchor (&) and alias (*) in the YAML spec.
  • Support multiple yaml documents to C# collection.
  • Customization
    • Rename key
    • Ignore member
• Mainly focused on Unity
  • Only 2021.3 and higher (netstandard2.1 compatible)

Most recent roadmap

• Support incremental source generator (Only Roslyn 4)
• Restrict max depth

Installation

NuGet

Require netstandard2.1 or later.

You can install the following nuget package. https://www.nuget.org/packages/VYaml

dotnet add package VYaml

Unity

Require Unity 2021.3 or later.

Install via git url

You can add following url to Unity Package Manager.

https://github.com/hadashiA/VYaml.git?path=VYaml.Unity/Assets/VYaml#0.13.1

Usage

Serialize / Deserialize

Define a struct or class to be serialized and annotate it with the [YamlObject] attribute and the partial keyword.

using VYaml.Annotations;

[YamlObject]
public partial class Sample
{
    // By default, public fields and properties are serializable.
    public string A; // public field
    public string B { get; set; } // public property
    public string C { get; private set; } // public property (private setter)
    public string D { get; init; } // public property (init-only setter)

    // use `[YamlIgnore]` to remove target of a public member
    [YamlIgnore]
    public int PublicProperty2 => PublicProperty + PublicField;
}

Why partial is necessary ?

• VYaml uses SourceGenerator for metaprogramming, which supports automatic generation of partial declarations, sets to private fields.

var utf8Yaml = YamlSerializer.Serialize(new Sample
{
    A = "hello",
    B = "foo",
    C = "bar",
    D = "hoge",
});

Result:

a: hello
b: foo
c: bar
d: hoge

By default, The Serialize<T> method returns an utf8 byte array. This is because it is common for writes to files or any data stores to be stored as strings in utf8 format.

If you wish to receive the results in a C# string, do the following Note that this has the overhead of conversion to utf16.

var yamlString = YamlSerializer.SerializeToString(...);

You can also convert yaml to C#.

using var stream = File.OpenRead("/path/to/yaml");
var sample = await YamlSerializer.DeserializeAsync<Sample>(stream);

// Or 
// var yamlUtf8Bytes = System.Text.Encofing.UTF8.GetBytes("<yaml string....>");
// var sample = YamlSerializer.Deserialize<Sample>(yamlUtf8Bytes);

sample.A // #=> "hello"
sample.B // #=> "foo"
sample.C // #=> "bar"
sample.D // #=> "hoge"

Built-in supported types

These types can be serialized by default:

• .NET primitives (byte, int, bool, char, double, etc.)
• Any enum (Currently, only simple string representation)
• string, decimal, Half
• TimeSpan, DateTime, DateTimeOffset
• Guid, Uri
• byte[] as base64 string
• T[]
• Nullable<>, KeyValuePair<,>, Tuple<,...>, ValueTuple<,...>
• List<>
• Dictionary<,>
• IEnumerable<>, ICollection<>, IList<>, IReadOnlyCollection<>, IReadOnlyList<>
• IDictionary<,>, IReadOnlyDictionary<,>

TODO: We plan add more.

Deserialize as dynamic

You can also deserialize into primitive object type implicitly.

var yaml = YamlSerializer.Deserialize<dynamic>(yamlUtf8Bytes);

yaml["a"] // #=> "hello"
yaml["b"] // #=> "aaa"
yaml["c"] // #=> "hoge"
yaml["d"] // #=> "ddd"

Deserialize multiple documents

YAML allows for multiple data in one file by separating them with ---. This is called a "Document". If you want to load multiple documents, you can use Yamlserializer.DeserializeMultipleDocuments<T>(...).

For example:

---
Time: 2001-11-23 15:01:42 -5
User: ed
Warning:
  This is an error message
  for the log file
---
Time: 2001-11-23 15:02:31 -5
User: ed
Warning:
  A slightly different error
  message.
---
Date: 2001-11-23 15:03:17 -5
User: ed
Fatal:
  Unknown variable "bar"
Stack:
- file: TopClass.py
  line: 23
  code: |
    x = MoreObject("345\n")
- file: MoreClass.py
  line: 58
  code: |-
    foo = bar

var documents = YamlSerializer.DeserializeMultipleDocuments<dynamic>(yaml);

documents[0]["Warning"] // #=> "This is an error message for the log file"
documents[1]["Warning"] // #=> "A slightly different error message."
documents[2]["Fatal"]   // #=> "Unknown variable \"bar\""

Naming convention

❗ By default, VYaml maps C# property names in lower camel case (e.g. propertyName) format to yaml keys.

You can customize this behaviour with [YamlMember("name")]

[YamlObject]
public partial class Sample
{
    [YamlMember("foo-bar-alias")]
    public int FooBar { get; init; }
}

This serialize as:

foo-bar-alias: 100

Custom constructor

VYaml supports both parameterized and parameterless constructors. The selection of the constructor follows these rules.

• If there is [YamlConstructor], use it.
• If there is no explicit constructor use a parameterless one.
• If there is one constructor use it.
• If there are multiple constructors, then the [YamlConstructor] attribute must be applied to the desired constructor (the generator will not automatically choose one), otherwise the generator will emit an error.

:note: If using a parameterized constructor, all parameter names must match corresponding member names (case-insensitive).

[YamlObject]
public partial class Person
{
    public int Age { get; } 
    public string Name { get; }

    // You can use a parameterized constructor - parameter names must match corresponding members name (case-insensitive)
    public Person(int age, string name)
    {
        Age = age;
        Name = name;
    }
}

[YamlObject]
public partial class Person
{
    public int Age { get; set; }
    public string Name { get; set; }
    
    public Person()
    {
        // ...
    }

    // If there are multiple constructors, then [YamlConstructor] should be used
    [YamlConstructor]
    public Person(int age, string name)
    {
        this.Age = age;
        this.Name = name;
    }
}


[YamlObject]
public partial class Person
{
    public int Age { get; } // from constructor
    public string Name { get; } // from constructor
    public string Profile { get; set; } // from setter

    // If all members of the construct are not taken as arguments, setters are used for the other members
    public Person3(int age, string name)
    {
        this.Age = age;
        this.Name = name;
    }
}

Enum

Deserialize a string in camelCase format as an enum.

enum Foo
{
    Item1,
    Item2,
    Item3,
}

YamlSerializer.Serialize(Foo.Item1); // #=> "item1"

It respect [EnumMember], and [DataMember].

enum Foo
{
    [EnumMember(Value = "item1-alias")]
    Item1,
    
    [EnumMember(Value = "item2-alias")]
    Item2,
    
    [EnumMember(Value = "item3-alias")]
    Item3,
}

YamlSerializer.Serialize(Foo.Item1); // #=> "item1-alias"

Polymorphism (Union)

VYaml supports deserialize interface or abstract class objects for. In VYaml this feature is called Union. Only interfaces and abstracts classes are allowed to be annotated with [YamlObjectUnion] attributes. Unique union tags are required.

[YamlObject]
[YamlObjectUnion("!foo", typeof(FooClass))]
[YamlObjectUnion("!bar", typeof(BarClass))]
public partial interface IUnionSample
{
}

[YamlObject]
public partial class FooClass : IUnionSample
{
    public int A { get; set; }
}

[YamlObject]
public partial class BarClass : IUnionSample
{
    public string? B { get; set; }
}

// We can deserialize as interface type.
var obj = YamlSerializer.Deserialize<IUnionSample>(UTF8.GetBytes("!foo { a: 100 }"));

obj.GetType(); // #=> FooClass

In the abobe example, The !foo and !bar are called tag in the YAML specification. YAML can mark arbitrary data in this way, and VYaml Union takes advantage of this.

You can also serialize:

YamlSerializer.Serialize<IUnionSample>(new FooClass { A = 100 });

Result:

!foo
a: 100

Customize serialization behaviour

• IYamlFormatter<T> is an interface customize the serialization behaviour of a your particular type.
• IYamlFormatterResolver is an interface can customize how it searches for IYamlFormatter<T> at runtime.

To perform Serialize/Deserialize, it need an IYamlFormatter<T> corresponding to a certain C# type.
By default, the following StandardResolver works and identifies IYamlFormatter.

You can customize this behavior as follows:

var options = new YamlSerializerOptions
{
    Resolver = CompositeResolver.Create(
        new IYamlFormatter[]
        {
            new YourCustomFormatter1(), // You can add additional formatter
        },
        new IYamlFormatterResolver[]
        {
            new YourCustomResolver(),  // You can add additional resolver
            StandardResolver.Instance, // Fallback to default behavior at the end.
        })
};
        
YamlSerializer.Deserialize<T>(yaml, options);
YamlSerializer.Deserialize<T>(yaml, options);

Low-Level API

Parser

YamlParser struct provides access to the complete meta-information of yaml.

• YamlParser.Read() reads through to the next syntax on yaml. (If end of stream then return false.)
• YamlParser.ParseEventType indicates the state of the currently read yaml parsing result.
• How to access scalar value:
  • YamlParser.GetScalarAs* families take the result of converting a scalar at the current position to a specified type.
  • YamlParser.TryGetScalarAs* families return true and take a result if the current position is a scalar and of the specified type.
  • YamlParser.ReadScalarAs* families is similar to GetScalarAs*, but advances the present position to after the scalar read.
• How to access meta information:
  • YamlParser.TryGetTag(out Tag tag)
  • YamlParser.TryGetCurrentAnchor(out Anchor anchor)

Basic example:

using var parser = YamlParser.FromBytes(utf8Bytes);

// YAML contains more than one `Document`. 
// Here we skip to before first document content.
parser.SkipAfter(ParseEventType.DocumentStart);

// Scanning...
while (parser.Read())
{
    // If the current syntax is Scalar, 
    if (parser.CurrentEventType == ParseEventType.Scalar)
    {
        var intValue = parser.GetScalarAsInt32();
        var stringValue = parser.GetScalarAsString();
        // ...
        
        if (parser.TryGetCurrentTag(out var tag))
        {
            // Check for the tag...
        }
        
        if (parser.TryGetCurrentAnchor(out var anchor))
        {
            // Check for the anchor...
        }        
    }
    
    // If the current syntax is Sequence (Like a list in yaml)
    else if (parser.CurrentEventType == ParseEventType.SequenceStart)
    {
        // We can check for the tag...
        // We can check for the anchor...
        
        parser.Read(); // Skip SequenceStart

        // Read to end of sequence
        while (!parser.End && parser.CurrentEventType != ParseEventType.SequenceEnd)
        {
             // A sequence element may be a scalar or other...
             if (parser.CurrentEventType = ParseEventType.Scalar)
             {
                 // ...
             }
             // ...
             // ...
             else
             {
                 // We can skip current element. (It could be a scalar, or alias, sequence, mapping...)
                 parser.SkipCurrentNode();
             }
        }
        parser.Read(); // Skip SequenceEnd.
    }
    
    // If the current syntax is Mapping (like a Dictionary in yaml)
    else if (parser.CurrentEventType == ParseEventType.MappingStart)
    {
        // We can check for the tag...
        // We can check for the anchor...
        
        parser.Read(); // Skip MappingStart

        // Read to end of mapping
        while (!parser.End && parser.CurrentEventType != ParseEventType.MappingEnd)
        {
             // After Mapping start, key and value appear alternately.
             
             var key = parser.ReadScalarAsString();  // if key is scalar
             var value = parser.ReadScalarAsString(); // if value is scalar
             
             // Or we can skip current key/value. (It could be a scalar, or alias, sequence, mapping...)
             // parser.SkipCurrentNode(); // skip key
             // parser.SkipCurrentNode(); // skip value
        }
        parser.Read(); // Skip MappingEnd.
    }
    
    // Alias
    else if (parser.CurrentEventType == ParseEventType.Alias)
    {
        // If Alias is used, the previous anchors must be holded somewhere.
        // In the High level Deserialize API, `YamlDeserializationContext` does exactly this. 
    }
}

See test code for more information. The above test covers various patterns for the order of ParsingEvent.

Emitter

Utf8YamlEmitter struct provides to write YAML formatted string.

Basic usage:

var buffer = new ArrayBufferWriter();
using var emitter = new Utf8YamlEmitter(buffer); // It needs buffer implemented `IBufferWriter<byte>`

emitter.BeginMapping(); // Mapping is a collection like Dictionary in YAML
{
    emitter.WriteString("key1");
    emitter.WriteString("value-1");
    
    emitter.WriteString("key2");
    emitter.WriteInt32(222);
    
    emitter.WriteString("key3");
    emitter.WriteFloat(3.333f);
}
emitter.EndMapping();

// If you want to expand a string in memory, you can do this.
System.Text.Encoding.UTF8.GetString(buffer.WrittenSpan); 

key1: value-1
key2: 222
key3: 3.333

Emit string in various formats

By default, WriteString() automatically determines the format of a scalar.

Multi-line strings are automatically format as a literal scalar:

emitter.WriteString("Hello,\nWorld!\n");

|
  Hello,
  World!

Special characters contained strings are automatically quoted.

emitter.WriteString("&aaaaa ");

"&aaaaa "

Or you can specify the style explicitly:

emitter.WriteString("aaaaaaa", ScalarStyle.Literal);

|-
  aaaaaaaa

Emit sequences and other structures

e.g:

emitter.BeginSequence();
{
    emitter.BeginSequence(SequenceStyle.Flow);
    {
        emitter.WriteInt32(100);
        emitter.WriteString("&hoge");
        emitter.WriteString("bra");
    }
    emitter.EndSequence();
    
    emitter.BeginMapping();
    {
        emitter.WriteString("key1");
        emitter.WriteString("item1");
        
        emitter.WriteString("key2");
        emitter.BeginSequence();
        {
            emitter.WriteString("nested-item1")
            emitter.WriteString("nested-item2")
            emitter.BeginMapping();
            {
                emitter.WriteString("nested-key1")
                emitter.WriteInt32(100)
            }
            emitter.EndMapping();
        }
        emitter.EndSequence();
    }
    emitter.EndMapping();
}
emitter.EndMapping();

- [100, "&hoge", bra]
- key1: item1
  key2:
  - nested-item1
  - nested-item2
  - nested-key1: 100

YAML 1.2 spec support status

Implicit primitive type conversion of scalar

The following is the default implicit type interpretation.

Basically, it follows YAML Core Schema. https://yaml.org/spec/1.2.2/#103-core-schema

https://yaml.org/spec/1.2.2/

Following is the results of the test for the examples from the yaml spec page.

• 2.1. Collections
  • ✅ Example 2.1 Sequence of Scalars (ball players)
  • ✅ Example 2.2 Mapping Scalars to Scalars (player statistics)
  • ✅ Example 2.3 Mapping Scalars to Sequences (ball clubs in each league)
  • ✅ Example 2.4 Sequence of Mappings (players’ statistics)
  • ✅ Example 2.5 Sequence of Sequences
  • ✅ Example 2.6 Mapping of Mappings
• 2.2. Structures
  • ✅ Example 2.7 Two Documents in a Stream (each with a leading comment)
  • ✅ Example 2.8 Play by Play Feed from a Game
  • ✅ Example 2.9 Single Document with Two Comments
  • ✅ Example 2.10 Node for “Sammy Sosa” appears twice in this document
  • ✅ Example 2.11 Mapping between Sequences
  • ✅ Example 2.12 Compact Nested Mapping
• 2.3. Scalars
  • ✅ Example 2.13 In literals, newlines are preserved
  • ✅ Example 2.14 In the folded scalars, newlines become spaces
  • ✅ Example 2.15 Folded newlines are preserved for “more indented” and blank lines
  • ✅ Example 2.16 Indentation determines scope
  • ✅ Example 2.17 Quoted Scalars
  • ✅ Example 2.18 Multi-line Flow Scalars
• 2.4. Tags
  • ⚠️ Example 2.19 Integers
  • ✅ Example 2.20 Floating Point
  • ✅ Example 2.21 Miscellaneous
  • ✅ Example 2.22 Timestamps
  • ✅ Example 2.23 Various Explicit Tags
  • ✅ Example 2.24 Global Tags
  • ✅ Example 2.25 Unordered Sets
  • ✅ Example 2.26 Ordered Mappings
• 2.5. Full Length Example
  • ✅ Example 2.27 Invoice
  • ✅ Example 2.28 Log File
• 5.2. Character Encodings
  • ✅ Example 5.1 Byte Order Mark
  • ✅ Example 5.2 Invalid Byte Order Mark
• 5.3. Indicator Characters
  • ✅ Example 5.3 Block Structure Indicators
  • ✅ Example 5.4 Flow Collection Indicators
  • ✅ Example 5.5 Comment Indicator
  • ✅ Example 5.6 Node Property Indicators
  • ✅ Example 5.7 Block Scalar Indicators
  • ✅ Example 5.8 Quoted Scalar Indicators
  • ✅ Example 5.9 Directive Indicator
  • ✅ Example 5.10 Invalid use of Reserved Indicators
• 5.4. Line Break Characters
  • ✅ Example 5.11 Line Break Characters
  • ✅ Example 5.12 Tabs and Spaces
  • ✅ Example 5.13 Escaped Characters
  • ✅ Example 5.14 Invalid Escaped Characters
• 6.1. Indentation Spaces
  • ✅ Example 6.1 Indentation Spaces
  • ✅ Example 6.2 Indentation Indicators
• 6.2. Separation Spaces
  • ✅ Example 6.3 Separation Spaces
• 6.3. Line Prefixes
  • ✅ Example 6.4 Line Prefixes
• 6.4. Empty Lines
  • ✅ Example 6.5 Empty Lines
• 6.5. Line Folding
  • ✅ Example 6.6 Line Folding
  • ✅ Example 6.7 Block Folding
  • ✅ Example 6.8 Flow Folding
• 6.6. Comments
  • ✅ Example 6.9 Separated Comment
  • ✅ Example 6.10 Comment Lines
  • ✅ Example 6.11 Multi-Line Comments
• 6.7. Separation Lines
  • ✅ Example 6.12 Separation Spaces
• 6.8. Directives
  • ✅ Example 6.13 Reserved Directives
  • ✅ Example 6.14 “YAML” directive
  • ✅ Example 6.15 Invalid Repeated YAML directive
  • ✅ Example 6.16 “TAG” directive
  • ✅ Example 6.17 Invalid Repeated TAG directive
  • ✅ Example 6.18 Primary Tag Handle
  • ✅ Example 6.19 Secondary Tag Handle
  • ✅ Example 6.20 Tag Handles
  • ✅ Example 6.21 Local Tag Prefix
  • ✅ Example 6.22 Global Tag Prefix
• 6.9. Node Properties
  • ✅ Example 6.23 Node Properties
  • ✅ Example 6.24 Verbatim Tags
  • ✅ Example 6.25 Invalid Verbatim Tags
  • ✅ Example 6.26 Tag Shorthands
  • ✅ Example 6.27 Invalid Tag Shorthands
  • ✅ Example 6.28 Non-Specific Tags
  • ✅ Example 6.29 Node Anchors
• 7.1. Alias Nodes
  • ✅ Example 7.1 Alias Nodes
• 7.2. Empty Nodes
  • ❌ Example 7.2 Empty Content
  • ✅ Example 7.3 Completely Empty Flow Nodes
• 7.3. Flow Scalar Styles
  • ✅ Example 7.4 Double Quoted Implicit Keys
  • ✅ Example 7.5 Double Quoted Line Breaks
  • ✅ Example 7.6 Double Quoted Lines
  • ✅ Example 7.7 Single Quoted Characters
  • ✅ Example 7.8 Single Quoted Implicit Keys
  • ✅ Example 7.9 Single Quoted Lines
  • ✅ Example 7.10 Plain Characters
  • ✅ Example 7.11 Plain Implicit Keys
  • ✅ Example 7.12 Plain Lines
• 7.4. Flow Collection Styles
  • ✅ Example 7.13 Flow Sequence
  • ✅ Example 7.14 Flow Sequence Entries
  • ✅ Example 7.15 Flow Mappings
  • ✅ Example 7.16 Flow Mapping Entries
  • ✅ Example 7.17 Flow Mapping Separate Values
  • ✅ Example 7.18 Flow Mapping Adjacent Values
  • ✅ Example 7.20 Single Pair Explicit Entry
  • ❌ Example 7.21 Single Pair Implicit Entries
  • ✅ Example 7.22 Invalid Implicit Keys
  • ✅ Example 7.23 Flow Content
  • ✅ Example 7.24 Flow Nodes
• 8.1. Block Scalar Styles
  • ✅ Example 8.1 Block Scalar Header
  • ❌ Example 8.2 Block Indentation Indicator
  • ✅ Example 8.3 Invalid Block Scalar Indentation Indicators
  • ✅ Example 8.4 Chomping Final Line Break
  • ✅ Example 8.5 Chomping Trailing Lines
  • ✅ Example 8.6 Empty Scalar Chomping
  • ✅ Example 8.7 Literal Scalar
  • ✅ Example 8.8 Literal Content
  • ✅ Example 8.9 Folded Scalar
  • ✅ Example 8.10 Folded Lines
  • ✅ Example 8.11 More Indented Lines
  • ✅ Example 8.12 Empty Separation Lines
  • ✅ Example 8.13 Final Empty Lines
  • ✅ Example 8.14 Block Sequence
  • ✅ Example 8.15 Block Sequence Entry Types
  • ✅ Example 8.16 Block Mappings
  • ✅ Example 8.17 Explicit Block Mapping Entries
  • ✅ Example 8.18 Implicit Block Mapping Entries
  • ✅ Example 8.19 Compact Block Mappings
  • ✅ Example 8.20 Block Node Types
  • ✅ Example 8.21 Block Scalar Nodes
  • ✅ Example 8.22 Block Collection Nodes

Credits

VYaml is inspired by:

• yaml-rust
• Utf8Json, MessagePack-CSharp, MemoryPack

Aurhor

@hadashiA

License

MIT