• Stars
    star
    716
  • Rank 63,241 (Top 2 %)
  • Language
    C#
  • License
    MIT License
  • Created about 8 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Compiles C# code by first rewriting the syntax trees of LINQ expressions using plain procedural code, minimizing allocations and dynamic dispatch.

roslyn-linq-rewrite

This tool compiles C# code by first rewriting the syntax trees of LINQ expressions using plain procedural code, minimizing allocations and dynamic dispatch.

Example input code

public int Method1()
{
    var arr = new[] { 1, 2, 3, 4 };
    var q = 2;
    return arr.Where(x => x > q).Select(x => x + 3).Sum();
}

Allocations: input array, array enumerator, closure for q, Where delegate, Select delegate, Where enumerator, Select enumerator.

Decompiled output code

public int Method1()
{
    int[] arr = new[] { 1, 2, 3, 4 };
    int q = 2;
    return this.Method1_ProceduralLinq1(arr, q);
}
private int Method1_ProceduralLinq1(int[] _linqitems, int q)
{
    if (_linqitems == null) throw new ArgumentNullException();

    int num = 0;
    for (int i = 0; i < _linqitems.Length; i++)
    {
        int num2 = _linqitems[i]; 
        if (num2 > q)
            num += num2 + 3;
    }
    return num;
}

Allocations: input array.

Note: in this example, the optimizing compiler has generated code using arrays, because the type was known at compile time. When this is not the case, the generated code will instead use a foreach and IEnumerable<>. Non-optimizable call chains and IQueryable<> are left intact.

Supported LINQ methods

  • Select, Where, Reverse, Cast, OfType
  • First, FirstOrDefault, Single, SingleOrDefault, Last, LastOrDefault
  • ToList, ToArray, ToDictionary
  • Count, LongCount, Any, All
  • ElementAt, ElementAtOrDefault
  • Contains, ForEach

Usage (.sln/.csproj)

  • Download the latest binaries
  • roslyn-linq-rewrite <path-to-sln-or-csproj> [--release]

Note: you can more deeply integrate the optimizing compiler with MSBuild by setting CscToolPath to the roslyn-linq-rewrite directory.

Usage (project.json)

  • Add the following to your project.json:
    "tools": {
        "dotnet-compile-csc-linq-rewrite": {
          "version": "1.0.1.9",
          "imports": "portable-net45+win8+wp8+wpa81"
        }
    }
  • In the buildOptions of your project.json, specify the custom compiler:
    "buildOptions": {
        "compilerName": "csc-linq-rewrite"
    }
  • Compile your project with dotnet restore and dotnet build.
  • If you need to exclude a specific method, apply a [NoLinqRewrite] attribute to that method:
namespace Shaman.Runtime
{
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct | AttributeTargets.Method)]
    public class NoLinqRewriteAttribute : Attribute
    {
    }
}

Tests

Here's a diff (and its code) between the ordinary LINQ tests and the version compiled with roslyn-linq-rewrite. The only difference seems to be related about a Min/Max with NaN values, I'm planning to fix it soon.

Shaman.FastLinq

To further reduce allocations, install Shaman.FastLinq or Shaman.FastLinq.Sources. These packages include LINQ methods specific for T[] and List<> (not all method calls are optimized by LINQ rewrite, like individual, non-chained .First() or .Last() calls).

Comparison to LinqOptimizer

  • Code is optimized at build time (as opposed to run time)
  • Uses existing LINQ syntax, no need for AsQueryExpr().Run()
  • No allocations for Expression<> trees and enumerator boxing
  • Parallel LINQ is not supported (i.e. left intact)
  • No support for F#

More Repositories

1

Shaman.Dokan.Archive

Mounts 7z/zip/rar files on Windows
C#
73
star
2

Shaman.ValueString

Allocation-free string-like type, string interning cache and allocation-free versions of StringBuilder/TextWriter.Write(int).
C#
30
star
3

Shaman.Reflection

Creates fast, strongly typed delegates for accessing methods, fields and properties. The right overload is automatically chosen based on the type arguments of Func<> or Action<>.
C#
21
star
4

Shaman.BlobStore

Provides an almost drop-in replacement for System.IO.File, optimized for a very large number of small files. Data is batched in package files, and can be read sequentially.
C#
20
star
5

Shaman.Dokan.Warc

Mounts WARC files on Windows
C#
16
star
6

ApiUnlock

C#
13
star
7

json-class-generator

JSON C#/TypeScript class generator
C#
13
star
8

Shaman.SingleThreadSynchronizationContext

Provides a single-threaded synchronization context, which makes it easy to reason about asynchronous code in environments that don't usually provide a synchronization context (for example, console applications).
C#
12
star
9

Shaman.Scraping

A C# library for reading/writing WARC files and scraping websites.
C#
7
star
10

Shaman.Stemming

Low-allocation version of Snowball stemmers.
C#
6
star
11

Shaman.Dom

An optimized version of HtmlAgilityPack with many bug fixes and improvements.
C#
6
star
12

Shaman.ProcessUtils

Provides wrappers for executing external processes, capturing their stdout, and automatically checking their exit code.
C#
5
star
13

Shaman.WinForms.IconHandler

Utilities for retrieving Windows Shell icons associated to a file.
C#
4
star
14

iabak-sharp

A C# implementation for the INTERNETARCHIVE.BAK project
C#
4
star
15

Shaman.Streams.Utf8

Utils for efficiently reading from streams of UTF8 text
C#
4
star
16

Shaman.DiffMatchPatch

Low-allocation version of DiffMatchPatch.
C#
4
star
17

Shaman.WinForms.Windows7Renderer

Windows 7-style ToolStrip renderer
C#
3
star
18

Shaman.System.Drawing

Compatibility layer for System.Drawing in CoreCLR.
C#
3
star
19

Shaman.WinForms.NaturalScroll

Redirects wheel scroll events to the control under the mouse, instead of the focused one.
C#
3
star
20

Shaman.CommonExtensions

Commonly used extension methods
C#
3
star
21

Shaman.JsonFile

Provides methods for quickly saving data structures to files (JSON or ProtoBuf), and methods for displaying lists/objects as tables/matrices.
C#
3
star
22

Shaman.FileSystem

Provides utility functions for generating unique file names and creating temporary files for transactional saves.
C#
2
star
23

RoslynLinqRewrite

Compiles a project by rewriting LINQ expressions into fast, procedural code with minimal allocations.
C#
2
star
24

Shaman.WinForms.VisualStudioRenderer

Visual Studio 2013-like ToolStripRenderer.
C#
2
star
25

epplus

C#
1
star
26

linqtests

C#
1
star
27

Shaman.WinForms.BetterLinkLabel

Provides a LinkLabel control with native hand cursor and Windows 7-like colors.
C#
1
star
28

Shaman.FileSize

Byte count/formatted file size parser and formatter.
C#
1
star
29

Shaman.TextTeaser

C# port of TextTeaser
C#
1
star
30

try_git

1
star
31

Shaman.Dokan.Base

Library that simplifies development of Dokan drives
C#
1
star
32

reddit-direct-links

Userscript that rewrites redd.it links as direct links
JavaScript
1
star
33

tinker

A clone of the Microsoft Tinker game
Pascal
1
star