• Stars
    star
    198
  • Rank 196,898 (Top 4 %)
  • Language
    F#
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 7 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

.NET Cross platform and highly composable middleware for building web request handlers in F#

Oryx

Build and Test codecov Nuget

Oryx is a high-performance .NET cross-platform functional HTTP request handler library for writing HTTP clients and orchestrating web requests in F#.

An SDK for writing HTTP web clients and orchestrating web requests.

This library enables you to write Web and REST clients and SDKs for various APIs and is currently used by the .NET SDK for Cognite Data Fusion (CDF).

Oryx is heavily inspired by the AsyncRx and Giraffe frameworks and applies the same ideas to the client making the web requests. You can think of Oryx as the client equivalent of Giraffe, where the HTTP request processing pipeline starting at the client, going all the way to the server and back again.

Installation

Oryx is available as a NuGet package. To install:

Using Package Manager:

Install-Package Oryx

Using .NET CLI:

dotnet add package Oryx

Or directly in Visual Studio.

Getting Started

open System.Net.Http
open System.Text.Json

open FSharp.Control.TaskBuilder

open Oryx
open Oryx.SystemTextJson.ResponseReader

[<Literal>]
let Url = "https://en.wikipedia.org/w/api.php"

let options = JsonSerializerOptions()

let query term = [
    struct ("action", "opensearch")
    struct ("search", term)
]

let asyncMain argv = task {
    use client = new HttpClient ()
    let request term =
        httpRequest
        |> GET
        |> withHttpClient client
        |> withUrl Url
        |> withQuery (query term)
        |> fetch
        |> json options
        
    let! result = request "F#" |> runAsync
    printfn "Result: %A" result
}

[<EntryPoint>]
let main argv =
    asyncMain().GetAwaiter().GetResult()
    0 // return an integer exit code

Fundamentals

The main building blocks in Oryx are the HttpContext and the HttpHandler. The context contains all the state needed for making the request, and also contains any response metadata such as headers, response code, etc received from the remote server:

type Context = {
    Request: HttpRequest
    Response: HttpResponse
}

The HttpContext is constructed using a pipeline of asynchronous HTTP handlers.

type IHttpNext<'TSource> =
    abstract member OnSuccessAsync: ctx: HttpContext * content: 'TSource -> Task<unit>
    abstract member OnErrorAsync: ctx: HttpContext * error: exn -> Task<unit>
    abstract member OnCancelAsync: ctx: HttpContext -> Task<unit>


type HttpHandler<'TSource> = IHttpNext<'TSource> -> Task<unit>

The relationship can be seen as:

do! handler success error cancel

An HTTP handler (HttpHandler) is a pipeline that uses or subscribes handler success error cancel the given continuations success, error and cancel, and return a Task of unit.

Each HttpHandler usually transforms the HttpRequest, HttpResponse or the content before passing it down the pipeline by invoking the next success continuation. It may also signal an error by invoking error with an exception to fail the processing of the pipeline.

The easiest way to get your head around the Oryx HttpHandler is to think of it as a functional web request processing pipeline. Each handler has the HttpContext and content at its disposal and can decide whether it wants to fail the request calling error, or continue the request by calling the success handler.

HTTP Handlers

The context and content may then be transformed for individual requests using a series of HTTP handlers. HTTP handlers are like lego bricks and may be composed into more complex HTTP handlers. The HTTP handlers included with Oryx are:

  • cache - Caches the last result of a given handler, both the context and the content.
  • catch - Catches errors and continues using another handler.
  • choose - Choose the first handler that succeeds in a list of handlers.
  • chunk - Chunks a sequence of HTTP handlers into sequential and concurrent batches.
  • concurrent - Runs a sequence of HTTP handlers concurrently.
  • empty - Creates a default empty request. You would usually start the chain with this handler.
  • fail- Fails the pipeline and pushes an exception downstream.
  • fetch - Fetches from remote using the current context
  • skip - Handler that skips (ignores) the content and outputs unit.
  • get - Retrieves the content (for use in http builder)
  • log - Log information about the given request.
  • map - Map the content of the HTTP handler.
  • panic - Fails the pipeline and pushes an exception downstream. This error cannot be catched or skipped.
  • parse - Parse response stream to a user-specified type synchronously.
  • parseAsync - Parse response stream to a user-specified type asynchronously.
  • sequential - Runs a sequence of HTTP handlers sequentially.
  • singleton - Handler that produces a single content value.
  • validate - Validate content using a predicate function.
  • withBearerToken - Adds an Authorization header with Bearer token.
  • withCancellationToken - Adds a cancellation token to use for the context. This is particularly useful when using Oryx together with C# client code that supplies a cancellation token.
  • withContent - Add HTTP content to the fetch request
  • withMetrics - Add and IMetrics interface to produce metrics info.
  • withError - Detect if the HTTP request failed, and then fail processing.
  • withHeader - Adds a header to the context.
  • withHeaders - Adds headers to the context.
  • withHttpClient - Adds the HttpClient to use for making requests using the fetch handler.
  • withHttpClientFactory - Adds an HttpClient factory function to use for producing the HttpClient.
  • withLogger - Adds an ILogger for logging requests and responses.
  • withLogLevel - The log level (LogLevel) that the logging should be performed at. Oryx will disable logging for LogLevel.None and this is also the default log level.
  • withLogFormat - Specify the log format of the log messages written.
  • withLogMessage - Log information about the given request supplying a user-specified message.
  • withMethod - with HTTP method. You can use GET, PUT, POST instead.
  • withQuery - Add URL query parameters
  • withResponseType - Sets the Accept header of the request.
  • withTokenRenewer - Enables refresh of bearer tokens without building a new context.
  • withUrl - Use the given URL for the request.
  • withUrlBuilder - Use the given URL builder for the request.
  • withUrlBuilder - Adds the URL builder to use. An URL builder constructs the URL for the Request part of the context.

In addition there are several extension for decoding JSON and Protobuf responses:

  • json - Decodes the given application/json response into a user-specified type.
  • protobuf - - Decodes the given application/protobuf response into a Protobuf specific type.

See JSON and Protobuf Content Handling for more information.

HTTP verbs

The HTTP verbs are convenience functions using the withMethod under the hood:

  • GET - HTTP get request
  • PUT - HTTP put request
  • POST - HTTP post request
  • DELETE - HTTP delete request
  • OPTIONS - HTTP options request

Composition

The real magic of Oryx is composition. The fact that everything is an HttpHandler makes it easy to compose HTTP handlers together. You can think of them as Lego bricks that you can fit together. Two or more HttpHandler functions may be composed together using the pipelining, i.e using the |> operator. This enables you to compose your web requests and decode the response, e.g as we do when listing Assets in the Cognite Data Fusion SDK:

    let list (query: AssetQuery) (source: HttpHandler<unit>) : HttpHandler<ItemsWithCursor<AssetReadDto>> =
        let url = Url +/ "list"

        source
        |> POST
        |> withVersion V10
        |> withResource url
        |> withContent (() -> new JsonPushStreamContent<AssetQuery>(query, jsonOptions))
        |> fetch
        |> withError decodeError
        |> json jsonOptions

The function list is now also an HttpHandler and may be composed with other handlers to create complex chains for doing multiple sequential or concurrent requests to a web service. And you can do this without having to worry about error handling.

Retrying Requests

Since Oryx is based on HttpClient from System.Net.Http, you may also use Polly for handling resilience.

Concurrent and Sequential Handlers

A sequential operator for running a list of HTTP handlers in sequence.

val sequential:
    handlers     : seq<HttpHandler<'TResult>>
                -> HttpHandler<list<'TResult>>

And a concurrent operator that runs a list of HTTP handlers in parallel.

val concurrent:
    handlers     : seq<HttpHandler<'TResult>>
                -> HttpHandler<list<'TResult>>

You can also combine sequential and concurrent requests by chunking the request. The chunk handler uses chunkSize and maxConcurrency to decide how much will be done in parallel. It takes a list of items and a handler that transforms these items into HTTP handlers. This is nice if you need to e.g read thousands of items from a web service in multiple requests.

val chunk:
   chunkSize     : int ->
   maxConcurrency: int ->
   handler       : (seq<'TSource> -> HttpHandler<seq<'TResult>>) ->
   items         : seq<'TSource>
                -> HttpHandler<seq<'TResult>>

Note that chunk will fail if one of the inner requests fails so for e.g a writing scenario you most likely want to create your own custom chunk operator that has different error semantics. If you write such operators then feel free to open a PR so we can include them in the library.

Error handling

To produce a custom error response you can use the withError handler after e.g fetch. The supplied errorHandler is given full access the the HttpResponse and the HttpContent and may produce a custom exception.

val withError:
   errorHandler  : (HttpResponse -> HttpContent -> Task<exn>) ->
   source        : HttpHandler<HttpContent> ->
   next          : IAsyncNext<HttpContext,HttpContent>
                -> Task<unit>

It's also possible to catch errors using the catch handler before e.g fetch. The function takes an errorHandler that is given the returned error and produces a new HttpHandler that may then decide to transform the error and continue processing or fail with an error. This is very helpful when a failed request not necessarily means an error, e.g if you need to check if an object with a given id exists at the server. It's not possible to catch a PanicException, so wrapping an exception in a PanicException can be used if you need to signal a fatal error and bypass a catch operator.

val catch:
   errorHandler  : (HttpContext -> exn -> HttpHandler<'TSource>) ->
   source        : HttpHandler<'TSource> ->
                -> HttpHandler<'TSource> ->

A choose operator takes a list of HTTP handlers and tries each of them until one of them succeeds. The choose operator will record every error that happens except for SkipException that can be used for skipping to the next handler. Other errors will be recorded. If multiple error happens they will be provided as an AggregateException. If you need break out of choose and force an exception without skipping to the next handler you can use the PanicException.

val choose:
   Handlers    : list<(HttpHandler<'TSource> ->HttpHandler<'TResult>)> ->
   source      : HttpHandler<'TSource>
              -> HttpHandler<'TResult>

JSON and Protobuf Content Handling

Oryx can serialize (and deserialize) content using:

System.Text.Json

Support for System.Text.Json is made available using the Oryx.SystemTextJson extension.

The json decode HTTP handler takes a JsonSerializerOptions to decode the response into user-defined type of 'T.

val json:
   options: JsonSerializerOptions
         -> HttpHandler<'TResult>

Content can be handled using type JsonPushStreamContent<'a> (content : 'T, options : JsonSerializerOptions).

Newtonsoft.Json

Support for Newtonsoft.Json is made available using the Oryx.NewtonsoftJson extension.

The json decode HTTP handler decodes the response into a user-defined type of 'TResult.

val json : HttpHandler<HttpContent,'TResult>

Content can be handled using type JsonPushStreamContent (content : JToken).

Thoth.Json.Net

Support for Thoth.Json.Net is made available using the Oryx.ThothJsonNet extension.

The json decoder takes a Decoder from Thoth.Json.Net to decode the response into a user-defined type of 'T.

val json:
   decoder: Decoder<'TResult>
         -> HttpHandler<'TResult>

Content can be handled using type JsonPushStreamContent (content : JsonValue).

Protobuf

Protobuf support is made available using the Oryx.Protobuf extension.

The protobuf decoder takes a Stream -> 'T usually generated by Google.Protobuf to decode the response into user defined type of 'T.

val protobuf: (System.IO.Stream -> 'TResult) ->  HttpHandler<System.Net.Http.HttpContent> -> HttpHandler<'TResult>

Both encode and decode uses streaming all the way so no large strings or arrays will be allocated in the process.

Content can be handled using type ProtobufPushStreamContent (content : IMessage).

Computational Expression Builder

Working with HttpContext objects can be a bit painful. To make it simpler to handle multiple requests using handlers you can use the req builder that will let you work with the content and hide the complexity of both the Context and the HttpNext.

http {
    let! assetDto = Assets.Entity.get key

    let asset = assetDto |> Asset.FromAssetReadDto
    if expands.Contains("Parent") && assetDto.ParentId.IsSome then
        let! parentDto = Assets.Entity.get assetDto.ParentId.Value
        let parent = parentDto |> Asset.FromAssetReadDto
        let expanded = { asset with Parent = Some parent }
        return expanded
    else
        return asset
}

The request may then be composed with other handlers, e.g chunked, retried, and/or logged.

To run a handler you can use the runAsync function.

val runAsync:
   handler: HttpHandler<'TResult>
         -> Task<Result<'TResult,exn>>

or the unsafe version that may throw exceptions:

val runUnsafeAsync:
   handler: HttpHandler<unit,'TResult>
         -> Task<'TResult>

Logging and Metrics

Oryx supports logging using the logging handlers. To setup for logging you first need to enable logging in the context by both setting a logger of type ILogger (Microsoft.Extensions.Logging) and the logging level to something higher than LogLevel.None.

val withLogger : (logger: ILogger) -> (context: EmptyContext) -> (context: EmptyContext)
val withLogLevel : (logLevel: LogLevel) -> (context: EmptyContext) -> (context: EmptyContext)
val withLogFormat (format: string) (context: EmptyContext) -> (context: EmptyContext)

The default format string is:

"Oryx: {Message} {HttpMethod} {Uri}\n{RequestContent}\n{ResponseContent}"

You can also use a custom log format string by setting the log format using withLogFormat. The available place holders you may use are:

  • Elapsed - The elapsed request time for fetch in milliseconds.
  • HttpMethod - The HTTP method used, i.e PUT, GET, POST, DELETE or PATCH.
  • Message - A user-supplied message using logWithMessage.
  • ResponseContent - The response content received. Must implement ToString to give meaningful output.
  • RequestContent - The request content being sent. Must implement ToString to give meaningful output.
  • ResponseHeader[key] - The response header received, replace key with the name of the header field.
  • Url - The URL used for fetching.

Note: Oryx will not call .ToString () but will hand it over to the ILogger for the actual string interpolation, given that the message will end up being logged.

NOTE: The logging handler (log) do not alter the types of the pipeline and may be composed anywhere. But to give meaningful output they should be composed after fetching (fetch). To log errors, the log handler should be placed after error handling (withError), and to log decoded responses the log handler should be placed after the decoder (i.e json).

val withLogger:
   logger: ILogger ->
   source: HttpHandler<'TSource>
        -> HttpHandler<'TSource>

val withLogLevel:
   logLevel: LogLevel ->
   source  : HttpHandler<'TSource>
          -> HttpHandler<'TSource>

val withLogMessage:
   msg : string              ->
   next: IHttpNext<'TSource>
      -> IHttpNext<'TSource>

val withLogMessage:
   msg   : string ->
   source: HttpHandler<'TSource>
        -> HttpHandler<'TSource>

Oryx may also emit metrics using the IMetrics interface (Oryx specific) that you can use with e.g Prometheus.

type IMetrics =
    abstract member Counter : metric: string -> labels: IDictionary<string, string> -> increase: int64 -> unit
    abstract member Gauge : metric: string -> labels: IDictionary<string, string> -> value: float -> unit

The currently defined Metrics are:

  • Metric.FetchInc - ("MetricFetchInc") The increase in the number of fetches when using the fetch handler.
  • Metric.FetchErrorInc - ("MetricFetchErrorInc"). The increase in the number of fetch errors when using the fetch handler.
  • Metrics.FetchRetryInc - ("MetricsFetchRetryInc"). The increase in the number of retries when using the retry handler.
  • Metric.FetchLatencyUpdate - ("MetricFetchLatencyUpdate"). The update in fetch latency (in milliseconds) when using the fetch handler.
  • Metric.DecodeErrorInc - ("Metric.DecodeErrorInc"). The increase in decode errors when using a json decode handler.

Labels are currently not set but are added for future use, e.g setting the error code for fetch errors etc.

Extending Oryx

It's easy to extend Oryx with your own HTTP handlers. Everything is functions, so you can easily add your own HTTP handlers.

Custom HTTP Handlers

Custom HTTP handlers may e.g populate the context, make asynchronous web requests and parse response content. HTTP handlers are functions that takes an HttpHandler'TSource>, and returns an HttpHandler<'TSource>. Example:

let withResource (resource: string) (source: HttpHandler<'TSource): HttpHandler<'TSource> =
    source
    |> update (fun ctx ->
        { ctx with
            Request =
                { ctx.Request with Items = ctx.Request.Items.Add(PlaceHolder.Resource, Value.String resource) } })
/// Parse response stream to a user specified type synchronously.
let parse<'TResult> (parser: Stream -> 'TResult) (source: HttpHandler<HttpContent>) : HttpHandler<'TResult> =
    fun next ->
        { new IHttpNext<HttpContent> with
            member _.OnSuccessAsync(ctx, content: HttpContent) =
                task {
                    let! stream = content.ReadAsStreamAsync()

                    try
                        let item = parser stream
                        return! next.OnSuccessAsync(ctx, item)
                    with
                    | ex ->
                        ctx.Request.Metrics.Counter Metric.DecodeErrorInc ctx.Request.Labels 1L
                        raise ex
                }

            member _.OnErrorAsync(ctx, exn) = next.OnErrorAsync(ctx, exn)
            member _.OnCancelAsync(ctx) = next.OnCancelAsync(ctx) }
        |> source

What is new in Oryx v5

Oryx v5 continues to simplify the HTTP handlers by reducing the number of generic parameters so you only need to specify the type the handler is producing (not what it's consuming). The HttpHandler have also been reduced to plain functions.

type IHttpNext<'TSource> =
    abstract member OnSuccessAsync: ctx: HttpContext * content: 'TSource -> Task<unit>
    abstract member OnErrorAsync: ctx: HttpContext * error: exn -> Task<unit>
    abstract member OnCancelAsync: ctx: HttpContext -> Task<unit>


type HttpHandler<'TSource> = IHttpNext<'TSource> -> Task<unit>

The great advantage is that you can now use the normal pipe operator (|>) instead of Kleisli composition (>=>). which will give you better type hinting and debugging in most IDEs.

use client = new HttpClient()

let common =
    httpRequest
    |> GET
    |> withHttpClient client
    |> withUrl Url
    |> cache

let! result =
    request common "F#"
    |> runUnsafeAsync
printfn $"Result: {result}"

let! result =
    request common "C#"
    |> runUnsafeAsync

What is new in Oryx v4

  • A validate handler has been added that can validate the passing content using a predicate function. If the predicate fails then the error path will be taken.

  • A protect handler has been added that protects the pipeline from exceptions (thrown upwards) and protocol error with regards to error / complete handling. E.g not allowed to call OnNextAsync() after OnErrorAsync().

  • The semantics of the choose operator have been modified so it continues processing the next handler if the current handler produces error i.e OnErrorAsync. Previously it was triggered by not calling .OnNextAsync()

  • Oryx v4 makes the content non-optional to simplify the HTTP handlers.

type IHttpNext<'TSource> =
    abstract member OnNextAsync: ctx: HttpContext * content: 'TSource -> Task<unit>
    abstract member OnErrorAsync: ctx: HttpContext * error: exn -> Task<unit>
    abstract member OnCompletedAsync: ctx: HttpContext -> Task<unit>

type HttpHandler<'TSource, 'TResult> =
    abstract member Subscribe: next: IHttpNext<'TResult> -> IHttpNext<'TSource>

type HttpHandler<'TSource> = HttpHandler<'TSource, 'TSource>

What is new in Oryx v3

Oryx v3 will significantly simplify the typing of HTTP handlers by:

  1. Be based on Async Observables instead of result returning continuations. The result returning continuations were problematic in the sense that they both push values down in addition to returning (pulling) async values up, thus each HTTP handler needed to care about the input (TSource), output (TNext), the final result (TResult) and error (TError) types. By never returning anything (Task<unit>) we get rid of the annoying return type.
  2. Error type ('TError) is now simply an exception (exn).
  3. Core logic refactored into a generic middleware (that can be reused for other purposes).

This change effectively makes Oryx an Async Observable (with context):

type IHttpNext<'TSource> =
    abstract member OnNextAsync: ctx: HttpContext * ?content: 'TSource -> Task<unit>
    abstract member OnErrorAsync: ctx: HttpContext * error: exn -> Task<unit>
    abstract member OnCompletedAsync: ctx: HttpContext -> Task<unit>

type IHttpHandler<'TSource, 'TResult> =
    abstract member Subscribe: next: IHttpNext<'TResult> -> IHttpNext<'TSource>

type IHttpHandler<'TSource> = IHttpHandler<'TSource, 'TSource>

The difference from observables is that the IHttpHandler subscribe method returns another "observer" (IHttpNext) instead of a Disposable and this observable is the side-effect that injects values into the pipeline (Subject). The composition stays exactly the same so all HTTP pipelines will works as before. The typing just gets simpler to handle.

The custom error type (TError) has also been removed and we now use plain exceptions for all errors. Any custom error types now needs to be an Exception subtype.

The retry operator has been deprecated. Use Polly instead. It might get back in a later release but the observable pattern makes it hard to retry something upstream.

A choose operator has been added. This operator takes a list of HTTP handlers and tries each of them until one of them succeeds.

What is new in Oryx v2

We needed to change Oryx to preserve any response headers and status-code that got lost after decoding the response content into a custom type. The response used to be a custom 'T so it could not hold any additional info. We changed this so the response is now an HttpResponse type:

type HttpResponse<'T> =
    {
        /// Response content
        Content: 'T
        /// Map of received headers
        Headers: Map<string, seq<string>>
        /// Http status code
        StatusCode: HttpStatusCode
        /// True if response is successful
        IsSuccessStatusCode: bool
        /// Reason phrase which typically is sent by servers together with the status code
        ReasonPhrase: string
    }

    /// Replaces the content of the HTTP response.
    member x.Replace<'TResult>(content: 'TResult): HttpResponse<'TResult> =
        {
            Content = content
            StatusCode = x.StatusCode
            IsSuccessStatusCode = x.IsSuccessStatusCode
            Headers = x.Headers
            ReasonPhrase = x.ReasonPhrase
        }

type Context<'T> =
    {
        Request: HttpRequest
        Response: HttpResponse<'T>
    }

Upgrade from Oryx v4 to v5

The context builders are gone. In Oryx v5 there is only HTTP handlers (HttpHandler). This means that there is only one way to build and transform the context. This might seem inefficient when you need to reuse the same part of the context for multiple requests. The way to handle this is to use the cache handler.

Upgrade from Oryx v3 to v4

The throw operator have been renamed to fail. The throw operator is still available but will give an obsoleted warning.

The content used through the handler pipeline is now non-optional. Thus custom code such as:

let withResource (resource: string): HttpHandler<'TSource> =
    { new IHttpHandler<'TSource, 'TResult> with
        member _.Subscribe(next) =
            { new IHttpNext<'TSource> with
                member _.OnNextAsync(ctx, ?content) =
                    next.OnNextAsync(
                        { ctx with
                            Request =
                                { ctx.Request with
                                    Items = ctx.Request.Items.Add(PlaceHolder.Resource, String resource)
                                }
                        },
                        ?content = content
                    )

                member _.OnErrorAsync(ctx, exn) = next.OnErrorAsync(ctx, exn)
                member _.OnCompletedAsync() = next.OnCompletedAsync()
            }}

Needs to be refactored to:

let withResource (resource: string): HttpHandler<'TSource> =
    { new HttpHandler<'TSource, 'TResult> with
        member _.Subscribe(next) =
            { new IHttpNext<'TSource> with
                member _.OnNextAsync(ctx, content) =
                    next.OnNextAsync(
                        { ctx with
                            Request =
                                { ctx.Request with
                                    Items = ctx.Request.Items.Add(PlaceHolder.Resource, String resource)
                                }
                        },
                        content = content
                    )

                member _.OnErrorAsync(ctx, exn) = next.OnErrorAsync(ctx, exn)
                member _.OnCompletedAsync() = next.OnCompletedAsync()
            }}

Upgrade from Oryx v2 to v3

Oryx v3 is mostly backwards compatible with v2. Your chains of operators will for most part look and work exactly the same. There are however some notable changes:

  • Context have been renamed to HttpContext.
  • HttpHandler have been renamed HttpHandler. This is because HttpHandler is now an interface.
  • The retry operator has been deprecated for now. Use Polly instead.
  • The catch operator needs to run after the error producing operator e.g fetch (not before). This is because Oryx v3 pushes results "down" instead of returning them "up" the chain of operators. The good thing with this change is that a handler can now continue processing the rest of the pipeline after catching an error. This was not possible in v2 / v1 where the catch operator had to abort processing and produce a result.
  • The log operator needs to be placed after the handler you want it to log. E.g to log JSON decoded data you need to place it after json.
  • Http handlers take 2 generic types instead of 4. E.g fetch<'TSource, 'TNext, 'TResult, 'TError> now becomes fetch<'TSource, 'TNext> and the last two types can simply be removed from your code.
  • ResponseError is gone. You need to sub-class an exception instead. This means that the `'TError' type is also gone from the handlers.
  • Custom context builders do not need any changes except renaming Context to HttpContext.
  • Custom HTTP handlers must be refactored. Instead of returning a result (Ok/Error) the handler needs to push down the result either using the Ok path next.OnNextAsync() or fail with an error next.OnErrorAsync(). This is very similar to e.g Reactive Extensions (Rx) OnNext / OnError. E.g:
 let withResource (resource: string) (next: NextFunc<_,_>) (context: HttpContext) =
    next { context with Request = { context.Request with Items = context.Request.Items.Add(PlaceHolder.Resource, String resource) } }

Needs to be refactored to:

let withResource (resource: string): HttpHandler<'TSource> =
    { new HttpHandler<'TSource, 'TResult> with
        member _.Subscribe(next) =
            { new IHttpNext<'TSource> with
                member _.OnNextAsync(ctx, ?content) =
                    next.OnNextAsync(
                        { ctx with
                            Request =
                                { ctx.Request with
                                    Items = ctx.Request.Items.Add(PlaceHolder.Resource, String resource)
                                }
                        },
                        ?content = content
                    )

                member _.OnErrorAsync(ctx, exn) = next.OnErrorAsync(ctx, exn)
                member _.OnCompletedAsync() = next.OnCompletedAsync()
            }}

It's a bit more verbose, but the hot path of the code is mostly the same as before.

Upgrade from Oryx v1 to v2

The context is now initiated with a content 'T of unit. E.g your custom HTTP handlers that is used before fetch need to be rewritten from using a 'TSource of HttpResponseMessage to unit e.g:

- let withLogMessage (msg: string) (next: HttpFunc<HttpResponseMessage, 'T, 'TError>) (context: EmptyContext) =
+ let withLogMessage (msg: string) (next: HttpFunc<unit, 'T, 'TError>) (context: EmptyContext) =

There is now also a runAsync' overload that returns the full HttpResponse record i.e: Task<Result<HttpResponse<'TResult>, HandlerError<'TError>>>. This makes it possible to get the response status-code, response-headers etc even after decoding of the content. This is great when using Oryx for a web-proxy or protocol converter where you need to pass on any response-headers.

Using Oryx with Giraffe

You can use Oryx within your Giraffe server if you need to make HTTP requests to other services. But then you must be careful about the order when opening namespaces so you know if you use the >=> operator from Oryx or Giraffe. Usually, this will not be a problem since the Giraffe >=> will be used within your e.g WebApp.fs or Server.fs, while the Oryx >=> will be used within the controller handler function itself e.g Controllers/Index.fs. Thus just make sure you open Oryx after Giraffe in the controller files.

open Giraffe
open Oryx

Libraries using Oryx:

Code of Conduct

This project follows https://www.contributor-covenant.org, see our Code of Conduct.

License

Apache v2, see LICENSE.

More Repositories

1

cognite-sdk-python

Cognite Python SDK
Python
76
star
2

reveal

Cognite Reveal 3D viewer
TypeScript
62
star
3

griff-react

High performance charting library for time series data
JavaScript
44
star
4

cognite-sdk-js

Cognite Javascript SDK
TypeScript
24
star
5

txid-syncing

Demo of continuous syncing based on txids
Python
18
star
6

cognite-sdk-dotnet

.NET SDK for Cognite Data Fusion (CDF)
C#
18
star
7

cdp-spark-datasource

Spark data source for Cognite Data Fusion
Scala
18
star
8

gearbox.js

GearBox is a place where application developers can contribute useful, reusable components across applications
TypeScript
15
star
9

three-combo-controls

Orbit & first person camera controller for THREE.JS
TypeScript
14
star
10

cognite-python-docs

Cognite examples and documentation for python.
13
star
11

python-extractor-utils

Framework for developing extractors in Python
Python
12
star
12

python-oidc-authentication

Sample code for using OIDC authentication against CDF
Python
10
star
13

open-industrial-data

Open workshop, hackathon and training materials based on the Open Industrial Data project
Jupyter Notebook
10
star
14

Steady-State-Detector

Jupyter Notebook
10
star
15

deploy-functions-oidc

A template repository to deploy functions using OIDC
Python
10
star
16

neat

Python
10
star
17

cognite-replicator

A package of scripts for replicating data between CDF tenants
Python
9
star
18

cognite-sdk-python-experimental

Cognite Python SDK Experimental Extensions
Python
8
star
19

opcua-extractor-net

OPC-UA extractor using libraries from OPCFoundation
C#
8
star
20

bazel-snapshots

A mechanism for doing incremental deploys with Bazel
Starlark
8
star
21

toolkit

Official Cognite Data Fusion Toolkit with project templates
Python
7
star
22

cognite-sdk-rust

Community Rust SDK for Cognite Data Fusion
Rust
7
star
23

processpool-python

Stable process pool library
Python
6
star
24

cognite-sdk-scala

Cognite Scala SDK
Scala
6
star
25

transformations-action-template

6
star
26

python-extractor-example

Example extractors of CSV and climate data using the Python extractor utils library
Python
6
star
27

victoriametrics2openmetrics

Simple converter from victoria metrics exports to open metrics expositions
Rust
6
star
28

cdffs

File System Interface for CDF Files
Python
5
star
29

discovery

An application focused around 3D, P&ID's and contextualization
TypeScript
5
star
30

fe-interview-challenge

starting create-react-app repo for front-end interview challenges
JavaScript
5
star
31

react-picture-annotation

Annotated Images and PDF Preview and Editor
TypeScript
5
star
32

cognite-model-hosting

Python
4
star
33

interacting-with-open-industrial-data

Jupyter Notebook
4
star
34

dotnet-extractor-utils

Common utilities for developing extractors in .NET
C#
4
star
35

data-model-examples

A repository containing data sets and examples for how to use Cognite Data Fusion, including data modeling capabilities
Python
4
star
36

inso-bootstrap-cli

CLI and GitHub-Action to configure and maintain CDF Projects (Groups, Data Sets, RAW DBs)
Python
4
star
37

publicdata

Jupyter Notebook
4
star
38

cdf-sdk-java

Java SDK for Cognite Data Fusion
Java
4
star
39

function-action-oidc

Python
3
star
40

solution-infra-local-dev

Setting up a local, K8-based dev environment for solution delivery
3
star
41

cognite-grafana-reveal

Show 3D models in grafana using https://github.com/cognitedata/reveal
TypeScript
3
star
42

datapoints-csv-extractor

An extractor utilized to extract datapoints stored in CSV format
Python
3
star
43

azure-digital-twin-cdf-sync

Python
3
star
44

power-ops-sdk

Python
3
star
45

cognite-python-utils

Python Utilities Extending Use of Cognite Products
Python
3
star
46

jetfire-cli

Command-line interface for Jetfire/Transformations
C#
3
star
47

learn-cognite-functions

Repository for the course material for Cognite Functions
Jupyter Notebook
3
star
48

python-extractor-utils-rest

REST extension for Cognite extractor-utils
Python
3
star
49

using-cognite-python-sdk

Code and materials for course on "Using Cognite Python SDK"
Jupyter Notebook
3
star
50

react-auth-wrapper

TypeScript
2
star
51

correlation

Package for discovering correlations in sensor data
Python
2
star
52

cognite-model-hosting-notebook

Python
2
star
53

transformations-template

A template that simplifies and robustifies deployment of SQL scripts to Cognite Transformations through CI/CD with the use of the jetfire CLI.
2
star
54

templates-docs

Docs for domains
2
star
55

indsl

InDSL - Industrial Data Science Library by Cognite
Python
2
star
56

auth-wrapper

[DEPRECATED] A OpenID Connect/OAuth 2.0 auth wrapper.
TypeScript
2
star
57

pygen

Python
2
star
58

transformations-cli

Python
2
star
59

dshub-tutorials

Templates for Cognite Data Studio
Jupyter Notebook
2
star
60

generator-create-react-redux-app

Generate react app with redux, styled components etc.
JavaScript
2
star
61

dotnet-simulator-utils

Utilities for developing simulator integrations within CDF
C#
2
star
62

TEKNA-WorkShop-PredictiveMaintenance

Jupyter Notebook
2
star
63

mqtt-extractor

MQTT extractor to CDF
Python
2
star
64

cdf-beam-connector-java

Cognite Data Fusion I/O connector for Apache Beam
Java
2
star
65

security-github-actions

Github Actions to facilitate continuous execution of security tools.
Python
2
star
66

reveal-viewer

WebGL viewer for high-performance visualization of large 3D models
2
star
67

datamodel-deploy-template

Template repository for deploying data models to CDF
Python
1
star
68

spark-jdbc-sink

A Spark metrics sink for databases using JDBC
Scala
1
star
69

python-extractor-utils-mqtt

MQTT extension for cognite python extractor utils.
Python
1
star
70

cdf-sdk-java-examples

Examples using the Java SDK with Cognite Data Fusion
Java
1
star
71

openfaas-dash-template

Dockerfile
1
star
72

storyhooks

A small utility to add lifecycle hooks to Storybooks for React components.
TypeScript
1
star
73

3d-web-downloader

JavaScript
1
star
74

interview-react-app

React boilerplate application
JavaScript
1
star
75

openctm-rs

A library for parsing OpenCTM files in Rust
Rust
1
star
76

well-domain-api-examples

Examples of ingestion into the Cognite Well Domain API
Python
1
star
77

edge-extractor

Edge extractor is remote data extraction agent
Go
1
star
78

custom-azure-key-vault-action

Python
1
star
79

i3df-specification

Specification and language wrappers for the I3DF format
Rust
1
star
80

functions-deploy-azure-pipelines

Sample azure pipelines for use with Cognite Functions
1
star
81

temp-frost-timeseries-extractor

Python
1
star
82

delivery-cdf-cli

CLI for working with CDF. Focusing on files primarily.
Java
1
star
83

inso-extpipes-cli

CLI and GitHub-Action to configure and maintain CDF Projects (Extraction-Pipelines). See README for instructions.
Python
1
star
84

ml-examples

Jupyter Notebook
1
star
85

function-action

Action for deploying a Cognite Function with a schedule
Python
1
star
86

enablement-bootcamp-config

1
star
87

cognite-synthetic-tags

Python
1
star
88

jetfire-doc

Documentation for Jetfire
1
star
89

reveal-demo

TypeScript
1
star
90

geospatial-examples

Examples for using Geospatial Service
Jupyter Notebook
1
star
91

bootcamp-functions

Template repository to get started with Cognite Functions during the CDF Bootcamp
Python
1
star
92

air-model-template-repo

The purpose of this repository is to provide a way for customers to interact with AIR and deploy models to CDF.
Shell
1
star
93

push-powerbi-tutorial

This is a simple example on how to push data from CDF to a Power BI dataset.
Jupyter Notebook
1
star
94

dshub-templates

Jupyter Notebook
1
star
95

cdf-oidc-migration

Scripts that can be used to migrate CDF tenants from Legacy authentication to Native Tokens authentication with Azure
PowerShell
1
star
96

sample-cdf-graphql-angular-app

Demo of how CDF SDK, ITG GraphQL and Reveal Viewer can work together in Angular
TypeScript
1
star
97

power-sdk

Cognite Power SDK
Python
1
star
98

bitcoin-extractor

Example of how to use the Coinmarketcap API and the Cognite SDK and APIs to store market data about cryptocurrencies
Python
1
star
99

cognite-async

Extensions to the python SDK for asynchronous operations
Python
1
star
100

eslint-config-cognite

[MOVED to the "applications" repository] Cognite eslint config e.g JS/ES styleguide
JavaScript
1
star