• Stars
    star
    12
  • Rank 1,542,971 (Top 32 %)
  • Language
    F#
  • License
    MIT License
  • Created over 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

TDMS 2.0 support for F# and C#

FSharp.Data.Tdms

FSharp.Data.Tdms provides support for TDMS 2.0 files<<#the-ni-tdms-file-format-entry,[1]>> from F# and C# on .NET 6 and later. From F# it allows you to access these in a type-safe manner through a generative type provider, while plain old functions and methods are available to both F# and C#. FSharp.Data.Tdms reads raw channel data from TDMS 2.0 files as arrays.

Missing features

  • Reading DAQmx raw data

  • Defragmenting TDMS files

  • Writing TDMS files

License

FSharp.Data.Tdms is licensed under an MIT License. Please refer to LICENSE for its full text.

Installation

.NET CLI

To add FSharp.Data.Tdms to a project using the .NET CLI, run the command:

dotnet add package FSharp.Data.Tdms --version 1.0.0-alpha.111

Be sure to replace 1.0.0-alpha.111 by the version you want to install.

Interactive

For interactive use, either in C# or F#, add the following directive to your script file or enter it into your REPL (e.g. dotnet fsi):

#r "nuget: FSharp.Data.Tdms, 1.0.0-alpha.111"

Be sure to replace 1.0.0-alpha.111 by the version you want to use.

Usage

From F#

Using the generative type provider, given a TDMS file Experiment.tdms with a channel containing floating-point values named Channel1 within a group named Group1:

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data

[<Literal>]
let Path = "Experiment.tdms"

type Experiment = TdmsProvider<Path, WriteIndex = true>

let experiment = Experiment(Path)

experiment.Group1.Channel1.Data |> Array.iter (printfn "%f")

Alternatively, use the functions in the FSharp.Data.Tdms namespace, mainly those in the File module:

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data.Tdms

File.read "Experiment.tdms" true
|> File.tryRawData<float> "Group1" "Channel1"
|> Option.defaultValue [||]
|> Array.iter (printfn "%f")

These functions have asynchronous equivalents:

#r "nuget: FSharp.Data.Tdms"

open FSharp.Data.Tdms

task {
    let! file = File.readAsync "Experiment.tdms" true
    let! data = File.tryRawDataAsync<float> "Group1" "Channel1"

    Option.defaultValue data
    |> Array.iter (printfn "%f")
}
|> Async.AwaitTask
|> Async.RunSynchronously

From C#

When using FSharp.Data.Tdms from C#, prefer the API for idiomatic C#:

using System;
using FSharp.Data.Tdms;

class Program
{
    static void Main(string[] args)
    {
        File.Read("Experiment.tdms", true).TryGetRawData("Group1", "Channel1", out double[] data);

        foreach (double sample in data)
        {
            Console.WriteLine(sample);
        }
    }
}

Or asynchronously:

using System;
using FSharp.Data.Tdms;

class Program
{
    static async Task Main(string[] args)
    {
        var file = await File.ReadAsync("Experiment.tdms", true);
        var data = await file.TryGetRawDataAsync("Group1", "Channel1");

        foreach (double sample in data)
        {
            Console.WriteLine(sample);
        }
    }
}

Property and raw data types

Most TDMS 2.0 data types directly map to .NET data types. The first exception is tdsTypeTimeStamp, which you can read as either FSharp.Data.Tdms.Timestamp (this data type corresponds to the NI LabVIEW timestamp), System.DateTime, System.DateTimeOffset, or System.TimeSpan. In case of System.TimeSpan, FSharp.Data.Tdms returns the time elapsed since 01/01/1904 00:00:00.00 UTC, as per this support document from NI.

The second exception is tdsTypeExtendedFloat. Since .NET do not support 80-bit extended precision floating point numbers, FSharp.Data.Tdms reads these as FSharp.Data.Tdms.Extended values.

Mapping from TDMS 2.0 to .NET data types in FSharp.Data.Tdms

Name

TDMS 2.0 data type

.NET data type

F# alias

C# alias

Void

tdsTypeVoid

FSharp.Core.Unit

unit

None

8-bit signed integer

tdsTypeI8

System.SByte

int8

sbyte

16-bit signed integer

tdsTypeI16

System.Int16

int16

short

32-bit signed integer

tdsTypeI32

System.Int32

int

int

64-bit signed integer

tdsTypeI64

System.Int64

int64

long

8-bit unsigned integer

tdsTypeU8

System.Byte

uint8

byte

16-bit unsigned integer

tdsTypeU16

System.UInt16

uint16

ushort

32-bit unsigned integer

tdsTypeU32

System.UInt32

uint

uint

64-bit unsigned integer

tdsTypeU64

System.UInt64

uint64

ulong

32-bit single-precision floating point

  • tdsTypeSingleFloat

  • tdsTypeSingleFloatWithUnit

System.Single

float32

float

64-bit double-precision floating point

  • tdsTypeDoubleFloat

  • tdsTypeDoubleFloatWithUnit

System.Double

float

double

80-bit extended-precision floating point

  • tdsTypeExtendedFloat

  • tdsTypeExtendedFloatWithUnit

FSharp.Data.Tdms.Extended

float80

None

Character string

tdsTypeString

System.String

string

string

Boolean

tdsTypeBoolean

System.Boolean

bool

bool

Timestamp

tdsTypeTimeStamp

None

None

32-bit single-precision floating point complex

tdsTypeComplexSingleFloat

System.ValueTuple<System.Single, System.Single>

struct (float32 * float32)

(float, float)

64-bit double-precision floating point complex

tdsTypeComplexDoubleFloat

System.Numerics.Complex

None

None

Performance

The BenchmarkDotNet benchmarks in this section give an idea of the performance of FSharp.Data.Tdms when compared to TDMSReader, the only other TDMS 2.0 implementation which works on .NET 6 and later. Since TDMSReader does not support reading TDMS index files, the benchmark disables this feature for FSharp.Data.Tdms as well, for a fair comparison. This means that FSharp.Data.Tdms may perform better in practice for TDMS files with many raw data segments.

Small file

This benchmark reads 30,489 double-precision floating points from a segmented 3.1 MB TDMS 2.0 file.

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK=7.0.101
  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG
  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0
Method Mean Error StdDev Ratio
TDMSReader 1,677.4 μs 3.55 μs 2.96 μs 1.00
FSharpDataTdms 741.4 μs 4.04 μs 3.78 μs 0.44
FSharpDataTdmsAsync 882.0 μs 13.53 μs 12.65 μs 0.52

Medium-sized file

This benchmark reads a channel of 43,200 strings from a segmented 138.1 MB TDMS 2.0 file.

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK=7.0.101
  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG
  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0
Method Mean Error StdDev Ratio
TDMSReader 8.570 s 0.0539 s 0.0505 s 1.00
FSharpDataTdms 2.581 s 0.0105 s 0.0098 s 0.30
FSharpDataTdmsAsync 2.691 s 0.0526 s 0.0516 s 0.31

Large file

This benchmark reads a channel of 779,297 double-precision floating points from a segmented 1.54 GB TDMS 2.0 file.

BenchmarkDotNet=v0.13.2, OS=opensuse-tumbleweed 20221223
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK=7.0.101
  [Host]   : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2 DEBUG
  .NET 7.0 : .NET 7.0.1 (7.0.122.56804), X64 RyuJIT AVX2

Job=.NET 7.0  Runtime=.NET 7.0
Method Mean Error StdDev Ratio
TDMSReader 996.0 ms 8.77 ms 7.77 ms 1.00
FSharpDataTdms 522.1 ms 5.81 ms 5.43 ms 0.52
FSharpDataTdmsAsync 654.5 ms 6.84 ms 6.07 ms 0.66

How to contribute

Imposter syndrome disclaimer: I want your help. No really, I do.

There might be a little voice inside that tells you you’re not ready; that you need to do one more tutorial, or learn another framework, or write a few more blog posts before you can help me with this project.

I assure you, that’s not the case.

And you don’t just have to write code. You can help out by writing documentation, tests, or even by giving feedback about this work. (And yes, that includes giving feedback about the contribution guidelines.)

Thank you for contributing!

References

<<#the-ni-tdms-file-format,[1]>> National Instruments. 2019. The NI TDMS File Format. (January 2019). Retrieved January 12, 2019 from http://www.ni.com/white-paper/3727/en/.