• Stars
    star
    573
  • Rank 74,737 (Top 2 %)
  • Language
    Objective-C
  • License
    Apache License 2.0
  • Created almost 6 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Twitter Network Layer is a scalable and feature rich network layer built on top of NSURLSession for Apple platforms

Twitter Network Layer (a.k.a TNL)

The Twitter Network Layer (TNL) is a framework for interfacing with the Apple provided NSURLSession stack that provides additional levels of control and insight over networking requests, provides simple configurability and minimizes the cognitive load necessary to maintain a robust and wide-reaching networking system.

OSI Layering with TNL

The Twitter Network Layer sits on top of the connection/session layer provided by the Apple NSURL framework. Those frameworks are build on top of the HTTP/1.1 and HTTP/2. The layer chart appears like this:

/--------------------------------------\
|                                      |
|              User Layer              |
|       The actual user (Layer 8)      |
|                                      |
|--------------------------------------|
|                                      |
|          Application Layer           |
|       MVC, MVVM, etc (Layer 7e)      |
|                                      |
|--------------------------------------|
|                                      |
|     Concrete Operation/App Layer     |  <------ Operations, Requests &
|             TNL (Layer 7d)           |          Responses built on TNL
|                                      |
|--------------------------------------|
|                                      |
|     Abstract Operation/App Layer     |
|             TNL (Layer 7c)           |  <------ TNL
|                                      |
|--------------------------------------|
|                                      |
|         Connection/App Layer         |
|        NSURL Stack (Layer 7b)        |
|                                      |
|--------------------------------------|
|                                      |
|          Protocol/App Layer          |
|     HTTP/1.1 & HTTP/2 (Layer 7a)     |
|                                      |
|--------------------------------------|
|                                      |
|            Presentation Layer        |
| Encryption & Serialization (Layer 6) |
|                                      |
|--------------------------------------|
|                                      |
|            Session Layer             |
|      A Feature of TCP (Layer 5)      |
|                                      |
|--------------------------------------|
|                                      |
|            Transport Layer           |
|             TCP (Layer 4)            |
|                                      |
|--------------------------------------|
|                                      |
|         Routing/Network Layer        |
|              IP (Layer 3)            |
|                                      |
|--------------------------------------|
|                                      |
|            Data Link Layer           |
|          IEEE 802.X (Layer 2)        |
|                                      |
|--------------------------------------|
|                                      |
|            Physical Layer            |
|          Ethernet (Layer 1)          |
|                                      |
\--------------------------------------/

Brief Overview

Features

Twitter Network Layer provides a framework on top of Apple's NSURLSession framework with numerous benefits. Here are some of the features provided by TNL:

  • All the features of NSURLSession, simplified where appropriate
  • NSOperation based request operations (for NSOperation features)
  • Strong separation of roles in the framework's objects
  • Immutable/mutable pairings of requests (TNLHTTPRequest) and configurations (TNLRequestConfiguration)
  • Encapsulated immutable responses (TNLResponse)
  • Prioritization of requests
  • Selectable response body consumption modes (NSData storage, callback chunking or saving to file)
  • Request hydration (enables polymorphic requests and dynamically populated requests)
  • Dynamic retrying with retry policies
  • More events (request operation state transitions, progress, network state/condition updates, etc)
  • Modular delegates for separation of roles and increased reusability and easy overriding

Usage

The high level concept of how to use TNL is rather straightforward:

  1. Set up any reuseable settings (by doing any combination of the following):
    • Build shared accessors to resuable TNLRequestConfiguration instances
    • Implement a TNLRequestDelegate (if added functionality is desired beyond just handling the result)
    • Configure a TNLRequestConfiguration for reuse
    • Configure the TNLGlobalConfiguration
  2. Set up any reusable TNLRequestOperationQueue objects once (ex: one for API requests, one for image requests, etc.)
    • [TNLRequestOperationQueue initWithIdentifier:]
  3. Generate and enqueue any desired TNLRequestOperation with the following objects:
    • TNLRequest conforming object (including TNLHTTPRequest concrete class and NSURLRequest)
    • TNLRequestConfiguration (optional)
    • TNLRequestDelegate (optional)
  4. Handle the events appropriately via the callbacks, particularly the completion callback that provides the TNLResponse
    • Delegate callbacks will go to the appropriate sub-protocol in the TNLRequestOperation's TNLRequestDelegate

HOWTO

Where To Start

Twitter Network Layer documentation starts with this README.md and progresses through the APIs via their documentation.

Overview of a Network Operation

Core Objects

The core objects of a service based architecture are request, response and operation/task/action (referred to as an operation from here on). The request encapsulates data to send and is not actionable; the response encapsulates the data received and is not actionable; and the operation is the object that delivers the request and retrieves the response and is the only actionable object in the set of core objects.

This high level concept translates directly into a network architecture as we will have requests that encapsulate the data of an HTTP request which are Headers and a Body, responses that encapsulate the data of an HTTP response which are Headers and a Body, and last the operation that executes delivering the request and retrieving the response.

Core Object Breakdown

  • request
    • encapsulates data to send
    • immutability provides stability
    • not actionable, just data
    • TNLRequest is the protocol for requests in TNL
    • TNLHTTPRequest and NSURLRequest are concrete classes (both are immutable/mutable pairs)
  • response
    • encapsulates data received
    • immutability provides stability
    • not actionable, just data
    • TNLResponse is the object for responses in TNL (composite object that includes an NSHTTPURLResponse)
  • operation
    • the executing object
    • delivers the request
    • retrieves the response
    • actionable (e.g. starting, canceling, priotiziation, modifying dependencies)
    • TNLRequestOperation is the operation in TNL (subclasses NSOperation) and is backed by NSURLSessionTask

Support Objects

In addition to a service architecture having requests, operations and responses; support objects are often present that aid in the management of the executing operations, configuration of their behavior and delegation of decisions or events.

The configuration object encapsulates how an operation behaves. It will have no impact on what is sent in the operation (that's the request), but can modify how it is sent. For instance, the configuration can indicate a maximum duration that the operation can take before it should fail.

The delegate object provides the extensibility of on demand decision making when prudent as well as the delivery of events as the operation executes.

The manager object coordinates the execution of multiple operations within a logical grouping.

Support Object Breakdown

  • configuration
    • encapsulation of behavior settings
    • TNLRequestConfiguration is the config in TNL (applied per operation)
    • NSURLSessionConfiguration is the config in NSURL stack (applied per manager)
  • delegate
    • provides extensibility
    • has callbacks for on demand decisions
    • has callbacks for events as they occur
    • TNLRequestDelegate is the delegate in TNL (provided per operation)
    • NSURLSessionDelegate is the delegate in NSURL stack (provided per manager)
  • manager
    • coordinator of multiple operations
    • permits grouping of operations
    • TNLRequestOperationQueue is the manager object in TNL
    • NSURLSession is the manager object in NSURL stack

Note: You can already see there is a fundamental architecture difference between NSURLSession networking and Twitter Network Layer. The configuration and delegate per operation approach in TNL is much more scalable when dealing with dozens or hundreds of unique configurations and/or delegates given a plethora of requests and their needs. Coupling the configuration and/or delegate to the reusable manager object(s) is unwieldy and can lead to mistakes w.r.t. correct configuration and event on a per request basis.

Building a Request

TNL uses the TNLRequest as the interface for all network requests. In practice, the protocol is used in one of 3 ways:

  1. Concrete TNLHTTPRequest
  • Configuring a concrete TNLHTTPRequest object (or TNLMutableHTTPRequest)
  1. NSURLRequest
  • NSURLRequest explicitely conforms to TNLRequest protocol via a category in TNL making it supported as request object.
  • However, since TNL rigidly segregates configuration from request, only the request properties on NSURLRequest are observed and the configuration properties of NSURLRequest are ignored.
  1. Implementing a custom TNLRequest
  • TNL supports having anything that conforms to TNLRequest as an original request for an operation.
  • Makes it simple for an object that encapsulates the minimal amount of information necessary to take the place as the original request.
    • You could have a APPRetrieveBlobRequest that has 1 property, the identifier for the "Blob" call blobIdentifier.
    • That object doesn't need to have any methods that actually represent anything related to an HTTP request and that's ok. However, in order for the operation to send the original request, it needs to be able to be treated as an HTTP request, which is to say it must conform to TNLRequest. This can be done in 2 ways:
      1. have the object implement TNLRequest and have its methods that populate the values by the relevant properties (in our example, the blob identifier)
      2. have the delegate implement the request hydration callback to convert the opaque request into a well formed TNLRequest ready for HTTP transport.
    • See Custom TNLRequest examples

When it comes to making the choice, it can boil down to convenience vs simplicity of reuse. If you have a one shot request that has no reuse, options 1 and 2 will suffice. If you have a request that can be reused throughout the code base, option 3 clearly offers the cleanest interface. By having the caller only need to know the class of the request and the relevant values for populating the request, any concern over the HTTP structure is completely eliminated.

Concrete TNLRequest with TNLHTTPRequest

TNLHTTPRequest:

NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
TNLHTTPRequest *request = [TNLHTTPRequest GETRequestWithURL:URL
                                           HTTPHeaderFields:@{@"User-Agent": [MYAPPDELEGATE userAgentString]}];

TNLMutableHTTPRequest:

NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
TNLMutableHTTPRequest *mRequest = [[TNLMutableHTTPRequest alloc] init];
mRequest.HTTPMethodValue = TNLHTTPMethodValueGET;
mRequest.URL = URL;
[mRequest setValue:[MYAPPDELEGATE userAgentString] forHTTPHeaderField:@"User-Agent"];

NSURLRequest

NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
NSMutableURLRequest *mRequest = [[NSMutableURLRequest alloc] init];
mRequest.HTTPMethod = @"GET";
mRequest.URL = URL;
[mRequest setValue:[MYAPPDELEGATE userAgentString] forHTTPHeaderField:@"User-Agent"];

Custom TNLRequest

1) Request Hydration

APPRetrieveBlobRequest *request = [[APPRetrieveBlobRequest alloc] initWithBlobIdentifier:blobIdentifier];

// ... elsewhere ...

- (void)tnl_requestOperation:(TNLRequestOperation *)op
              hydrateRequest:(APPRetrieveBlobRequest *)request // we know the type
                  completion:(TNLRequestHydrateCompletionBlock)complete
{
     NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobRequest.blobIdentifier];
     NSURL *URL = [NSURL URLWithString:URLString];
     TNLHTTPRequest *newReq = [TNLHTTPRequest GETRequestWithURL:URL
                                               HTTPHeaderFields:@{@"User-Agent": [MYAPPDELEGATE userAgentString]}];
     complete(newReq);
}

2) Request with HTTP support

APPRetrieveBlobRequest *request = [[APPRetrieveBlobRequest alloc] initWithBlobIdentifier:blobIdentifier];

// ... elsewhere ...

@implementation APPRetrieveBlobRequest

- (NSURL *)URL
{
    NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", self.blobIdentifier];
    return [NSURL URLWithString:URLString];
}

- (NSDictionary *)allHTTPHeaderFields
{
    return @{@"User-Agent":[MYAPPDELEGATE userAgentString]};
}

// POINT OF IMPROVEMENT:
// utilize polymorphism and have an APPBaseRequest class that implements
// the "allHTTPHeaderFields" so that all subclasses (including APPRetrieveBlobRequest)
// will inherit the desired defaults.
// This can apply to a wide range of HTTP related TNLHTTPRequest properties
// or even composition of subcomponents that are aggregated to a single property.
// For example: the host of the URL (api.myapp.com) could be provided
// as a property on APPBaseRequest that permits subclasses to override the host, and then
// the construction of the `URL` uses composition of variation properites that the subclasses
// can provide.

@end

Inspecting a Response

When an operation completes, a TNLResponse is populated and provided to the completion block or completion callback (depending on if you use a delegate or not). The TNLResponse has all the information necessary to understand how the operation completed, as well as what information was retrieve. Additionally, with response polymorphism, the response can be extended to provide better contextual information regarding the result, such as parsing the response body as JSON or converting the response body into a UIImage.

The way you deal with a TNLResponse should be systematic and straighforward:

  1. deal with any errors on the response
    • TNLResponse has an operationError property but custom subclasses could expose other errors too.
    • Subclass response objects that have extra errors should consider having an anyError property for quick access to any error in the response.
  2. deal with the status code of the response
    • It is important to know that a 404 is not an operation error so it won't be set as an error.
    • It is actually the status of the successful operation and needs to be handled accordingly.
    • For designs that want to treat HTTP Status codes that are not success as errors, they should expose an HTTP error on their response subclass(es).
  3. deal with the response payload
    • This could be the response HTTP headers, the response body (as NSData or a file on disk), etc
    • response subclasses should consider deserializing their response's body into a model object and exposing it as a property for concrete interactions.

One benefit to using response polymorphism is the ability to handle the response and populate the hydrated response with the information that's pertinent to the caller. For example: if your network operation yields JSON, and all you care about is if that JSON came through or not, at hydration time you could check for any error conditions then parse out the JSON and if everything is good have a property on the custom TNLResponse subclasss that holds the NSDictionary result property (or nil if anything along the way prevented success).

Things you can inspect on a response by default:

  • the operation error (if one occurred)
  • the original request
  • the response info
    • this object encapsulates the information of the HTTP response including:
      • the source of the response (local cache or network load)
      • the response body (as data or temporarySavedFile if the operation was configured to maintain the data)
      • the final NSURLRequest that loaded the response
      • the final NSURLResponse object
    • it also provides convenience accessors
      • the response's HTTP status code
      • the final NSURL
      • all the HTTP header fields
  • the response metrics
    • detailed metric information such as execution timings, durations, bytes sent/received, attempt counts, etc.
    • this is the detail that TNL exposes for every request/operation/response that really empowers programmers to maximize impact with their networking.

Simple Network Operations

Twitter Network Layer provides a highly robust API for building network operations with a great deal of flexibility and extensibility. However, there are often occasions when you just need to execute an operation and need things to be as simple as possible. Twitter Network Layer provides all the convenience necessary for getting what needs to be done as simply as possible.

 NSString *URLString = [NSURL URLWithString:@"http://api.myapp.com/settings"];
 NSURLRequest *request = [NSURLRequest requestWithURL:URLString];
 [[TNLRequestOperationQueue defaultOperationQueue] enqueueRequest:request
                                                       completion:^(TNLRequestOperation *op, TNLResponse *response) {
     NSDictionary *json = nil;
     if (!response.operationError && response.info.statusCode == 200) {
         json = [NSJSONSerialization JSONObjectWithData:response.info.data options:0 error:NULL];
     }
     if (json) {
        [self _myapp_didCompleteSettingsRequestWithJSON:json];
     } else {
        [self _myapp_didFailSettingsRequest];
     }
 }];

Configuring Behavior

Often the vanila configuration for an operation will suffice, however it is common to need particular behaviors in order to get specific use cases to work. Let's take, as an example, firing network operation when a specific metric is hit. In this case, we don't care about storing the response body and we also want to avoid having a cache that could get in the way.

 NSURL *URL = [NSURL URLWithString:@"http://api.myapp.com/recordMetric?hit=true"];
 TNLHTTPRequest *request = [TNLHTTPRequest GETRequestWithURL:URL HTTPHeaderFields:nil];
 TNLMutableRequestConfiguration *config = [TNLMutableRequestConfiguration defaultConfiguration];
 config.responseDataConsumptionMode = TNLResponseDataConsumptionModeNone;
 config.URLCache = nil; // Note: 'URLCache' is now 'nil' by default in TNL, but the illustration still works
 TNLRequestOperation *op = [TNLRequestOperation operationWithRequest:request
                                                       configuration:config
                                                          completion:^(TNLRequestOperation *o, TNLResponse *response) {
     assert(response.info.source != TNLResponseSourceLocalCache);
     const BOOL success = response.info.statusCode == 202;
     [self didSendMetric:success];
 }];
 [[TNLRequestOperationQueue defaultOperationQueue] enqueueOperation:op];

Now, sometimes, you may want to have the same defaults for certain kinds of operations. That can easily be accomplished with a category or some other shared accessor.

 @interface TNLRequestConfiguration (APPAdditions)
 + (instancetype)configurationForMetricsFiring;
 @end

 @implementation TNLRequestConfiguration (APPAdditions)

 + (instancetype)configurationForMetricsFiring
 {
     static TNLRequestConfiguration* sConfig;
     static dispatch_once_t onceToken;
     dispatch_once(&onceToken, ^{
         TNLMutableRequestConfiguration *mConfig = [TNLMutableRequestConfiguration defaultConfiguration];
         mConfig.URLCache = nil; // Note: 'URLCache' is now 'nil' by default in TNL, but the illustration still works
         mConfig.responseDataConsumptionMode = TNLResponseDataConsumptionModeNone;
         sConfig = [mConfig copy];
     });
     return sConfig;
 }

 @end

 @implementation TNLMutableRequestConfiguration (APPAdditions)

 + (instancetype)configurationForMetricsFiring
 {
     return [[TNLRequestConfiguration configurationForMetricsFiring] mutableCopy];
 }

 @end

Building an Advanced API Layer

Twitter Network Layer was designed from the ground up with REST APIs in mind. From simple APIs to complex API layers that require a complicated system for managing all operations, TNL provides the foundation needed.

As a pattern for creating concrete API operations, one of the first places to extend TNL for your API layer is by concretely building API requests and responses. For requests, you implement a TNLRequest for every request your API provides with properties that configure each request appropriately. Those requests should be subclassing a base request that does the busy work of pulling together the generic properties that the subclasses can override to construct the HTTP properties of the request. Each subclassed request then overrides only what is necessary to form the valid HTTP request. For things that are context or time sensitive, such as request signing, request hydration should be used to fully saturate the custom API request at the time the request is going to sent (vs at the time it was enqueued).

Following from custom API requests are custom API responses. At a minimum, it makes sense to have an API response that subclasses TNLResponse. To provide an even simpler interface to callers, you can implement a response per request. For response hydration, you merely extract whatever contextually relevant information is valuable for an API response and set those properties on you custom subclass of TNLResponse (such as API error, JSON result, etc).

If the API layer is advanced enough, it may warrant further encapsulation with a managing object which is often referred to as an API client. The API client would manage the queuing of requests, the delegate implementation for operations (including hydration for requests and subclassing responses so they hydrate too), the vending of operations, authentication/signing of requests, high level retry plus timeout behavior, custom configurations and oberving responses for custom handling.

Client Architecture

With an API client architecture, the entire HTTP structure is encapsulated and callers can deal with things just the objects they care about. No converting or validating. No configuring. The power of TNL is completely utilized by API client freeing the caller of any burden.

 APISendMessageRequest *request = [[APISendMessageRequest alloc] init];
 request.sender = self.user;
 request.receiver = otherUser;
 request.message = message;
 self.sendOp = [[APIClient sharedInstance] enqueueRequest:request
                                           completion:^(TNLRequestOperation *op, APIResponse *response) {
    [weakSelf messageSendDidComplete:op withResponse:(id)response];
 }];

 // ... elsewhere ...

 - (void)messageSendDidComplete:(TNLRequestOperation *)op withResponse:(APISendMessageResponse *)response
 {
     assert(self.sendOp == op);
     self.sendOp = nil;
     if (!sendMessageResponse.wasCancelled) {
        if (sendMessageResponse.succeeded) {
            [self updateUIForCompletedMessageSendWithId:sendMessageResponse.messageId];
        } else {
            [self updateUIForFailedMessageSendWithUserErrorTitle:sendMessageResponse.errorTitle
                                                    errorMessage:sendMessageResponse.errorMessage];
        }
     }
 }

 // Insight:
 // Presumably, APISendMessageResponse would subclass a base response like APIBaseResponse.
 // Following that presumption, it would make sense that APIBaseResponse would expose
 // wasCancelled, succeeded, errorTitle and errorMessage while APISendMessageResponse would
 // expose messageId (since that is part of the response payload that is specific to the request).
 // It would likely make sense that if the API used JSON response bodies,
 // the base response would also expose a "result" property (NSDictionary) and
 // APISendMessageResponse's implementation for messageId is just:
 //    return self.result[@"newMessageId"];

Using the Command-Line-Interface

Twitter Network Layer includes a target for building a macOS tool called tnlcli. You can build this tool run it from Terminal from you Mac, similar to cURL or other networking command line utilities.

Usage

Usage: tnlcli [options] url

    Example: tnlcli --request-method HEAD --response-header-mode file,print --response-header-file response_headers.json https://google.com

Argument Options:
-----------------

    --request-config-file <filepath>     TNLRequestConfiguration as a json file
    --request-headers-file <filepath>    json file of key-value-pairs for using as headers
    --request-body-file <filepath>       file for the HTTP body

    --request-header "Field: Value"      A header to provide with the request (will override the header if also in the request header file). Can provide multiple headers.
    --request-config "config: value"     A config setting for the TNLRequestConfiguration of the request (will override the config if also in the request config file). Can provide multiple configs.
    --request-method <method>            HTTP Method from Section 9 in HTTP/1.1 spec (RFC 2616), such as GET, POST, HEAD, etc

    --response-body-mode <mode>          "file" or "print" or a combo using commas
    --response-body-file <filepath>      file for the response body to save to (requires "file" for --response-body-mode
    --response-headers-mode <mode>       "file" or "print" or a combo using commas
    --response-headers-file <filepath>   file for the response headers to save to (as json)

    --dump-cert-chain-directory <dir>    directory for the certification chain to be dumped to (as DER files)

    --verbose                            Will print verbose information and force the --response-body-mode and --responde-headers-mode to have "print".
    --version                            Will print ther version information.

License

Copyright 2014-2020 Twitter, Inc.

Licensed under the Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0

Security Issues?

Please report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.

More Repositories

1

the-algorithm

Source code for Twitter's Recommendation Algorithm
Scala
60,968
star
2

twemoji

Emoji for everyone. https://twemoji.twitter.com/
HTML
16,575
star
3

typeahead.js

typeahead.js is a fast and fully-featured autocomplete library
JavaScript
16,526
star
4

twemproxy

A fast, light-weight proxy for memcached and redis
C
12,000
star
5

the-algorithm-ml

Source code for Twitter's Recommendation Algorithm
Python
9,844
star
6

finagle

A fault tolerant, protocol-agnostic RPC system
Scala
8,742
star
7

hogan.js

A compiler for the Mustache templating language
JavaScript
5,143
star
8

labella.js

Placing labels on a timeline without overlap.
JavaScript
3,869
star
9

scala_school

Lessons in the Fundamentals of Scala
HTML
3,700
star
10

AnomalyDetection

Anomaly Detection with R
R
3,529
star
11

scalding

A Scala API for Cascading
Scala
3,467
star
12

twitter-text

Twitter Text Libraries. This code is used at Twitter to tokenize and parse text to meet the expectations for what can be used on the platform.
HTML
3,051
star
13

TwitterTextEditor

A standalone, flexible API that provides a full-featured rich text editor for iOS applications.
Swift
2,950
star
14

opensource-website

Twitter's open source website, identifying projects we've released, organizations we support, and the work we do to support open source.
SCSS
2,918
star
15

util

Wonderful reusable code from Twitter
Scala
2,679
star
16

algebird

Abstract Algebra for Scala
Scala
2,284
star
17

finatra

Fast, testable, Scala services built on TwitterServer and Finagle
Scala
2,271
star
18

effectivescala

Twitter's Effective Scala Guide
HTML
2,241
star
19

summingbird

Streaming MapReduce with Scalding and Storm
Scala
2,136
star
20

pelikan

Pelikan is Twitter's unified cache backend
C
1,921
star
21

ios-twitter-image-pipeline

Twitter Image Pipeline is a robust and performant image loading and caching framework for iOS clients
C
1,851
star
22

twurl

OAuth-enabled curl for the Twitter API
Ruby
1,790
star
23

twitter-server

Twitter-Server defines a template from which services at Twitter are built
Scala
1,542
star
24

rezolus

Systems performance telemetry
Rust
1,541
star
25

activerecord-reputation-system

An Active Record Reputation System for Rails
Ruby
1,334
star
26

communitynotes

Documentation and source code powering Twitter's Community Notes
Python
1,319
star
27

compose-rules

Static checks to aid with a healthy adoption of Compose
Kotlin
1,311
star
28

fatcache

Memcache on SSD
C
1,301
star
29

rsc

Experimental Scala compiler focused on compilation speed
Scala
1,245
star
30

elephant-bird

Twitter's collection of LZO and Protocol Buffer-related Hadoop, Pig, Hive, and HBase code.
Java
1,137
star
31

cassovary

Cassovary is a simple big graph processing library for the JVM
Scala
1,039
star
32

Serial

Light-weight, fast framework for object serialization in Java, with Android support.
Java
988
star
33

hbc

A Java HTTP client for consuming Twitter's realtime Streaming API
Java
963
star
34

twemcache

Twemcache is the Twitter Memcached
C
925
star
35

innovators-patent-agreement

Innovators Patent Agreement (IPA)
919
star
36

vireo

Vireo is a lightweight and versatile video processing library written in C++11
C++
919
star
37

twitter-korean-text

Korean tokenizer
Scala
834
star
38

scrooge

A Thrift parser/generator
Scala
785
star
39

BreakoutDetection

Breakout Detection via Robust E-Statistics
C++
753
star
40

GraphJet

GraphJet is a real-time graph processing library.
Java
696
star
41

twitter-cldr-rb

Ruby implementation of the ICU (International Components for Unicode) that uses the Common Locale Data Repository to format dates, plurals, and more.
Ruby
667
star
42

bijection

Reversible conversions between types
Scala
657
star
43

chill

Scala extensions for the Kryo serialization library
Scala
607
star
44

hadoop-lzo

Refactored version of code.google.com/hadoop-gpl-compression for hadoop 0.20
Shell
544
star
45

storehaus

Storehaus is a library that makes it easy to work with asynchronous key value stores
Scala
464
star
46

rpc-perf

A tool for benchmarking RPC services
Rust
458
star
47

d3kit

D3Kit is a set tools to speed D3 related project development
JavaScript
429
star
48

scoot

Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution.
Go
347
star
49

twitter-cldr-js

JavaScript implementation of the ICU (International Components for Unicode) that uses the Common Locale Data Repository to format dates, plurals, and more. Based on twitter-cldr-rb.
JavaScript
345
star
50

scala_school2

Scala School 2
Scala
340
star
51

rustcommon

Common Twitter Rust lib
Rust
339
star
52

wordpress

The official Twitter plugin for WordPress. Embed Twitter content and grow your audience on Twitter.
PHP
310
star
53

ios-twitter-logging-service

Twitter Logging Service is a robust and performant logging framework for iOS clients
Objective-C
299
star
54

nodes

A library to implement asynchronous dependency graphs for services in Java
Java
246
star
55

SentenTree

A novel text visualization technique
JavaScript
226
star
56

interactive

Twitter interactive visualization
HTML
213
star
57

joauth

A Java library for authenticating HTTP Requests using OAuth
Java
211
star
58

thrift_client

A Thrift client wrapper that encapsulates some common failover behavior
Ruby
196
star
59

hpack

Header Compression for HTTP/2
Java
192
star
60

zktraffic

ZooKeeper protocol analyzer and stats gathering daemon
Python
165
star
61

twemoji-parser

A simple library for identifying emoji entities within a string in order to render them as Twemoji.
Scala
162
star
62

cache-trace

A collection of Twitter's anonymized production cache traces.
Shell
162
star
63

sbf

Java
159
star
64

tormenta

Scala extensions for Storm
Scala
132
star
65

whiskey

HTTP library for Android (beta)
Java
131
star
66

hraven

hRaven collects run time data and statistics from MapReduce jobs in an easily queryable format
Java
127
star
67

netty-http2

HTTP/2 for Netty
Java
120
star
68

sqrl

A Safe, Stateful Rules Language for Event Streams
TypeScript
100
star
69

ccommon

Cache Commons
C
99
star
70

focus

Focus aligns Git worktree content based on outlines of a repository's Bazel build graph. Focused repos are sparse, shallow, and thin and unlock markedly better performance in large repos.
Rust
91
star
71

dict_minimize

Access scipy optimizers from your favorite deep learning framework.
Python
77
star
72

metrics

76
star
73

twitter.github.io

HTML
71
star
74

go-bindata

Go
68
star
75

diffusion-rl

Python
66
star
76

birdwatch

64
star
77

cloudhopper-commons

Cloudhopper Commons
Java
57
star
78

twitter-cldr-npm

TwitterCldr npm package
JavaScript
49
star
79

.github

Twitter GitHub Organization-wide files
48
star
80

bazel-multiversion

Bazel rules to resolve, fetch and manage 3rdparty JVM dependencies with support for multiple parallel versions of the same dependency. Powered by Coursier.
Scala
47
star
81

libwatchman

A C interface to watchman
C
44
star
82

sslconfig

Twitter's OpenSSL Configuration
42
star
83

ios-twitter-apache-thrift

A thrift encoding and decoding library for Swift
Swift
41
star
84

gatekeeper-service

GateKeeper is a service built to automate the manual steps involved in onboarding, offboarding, and lost asset scenarios.
Python
36
star
85

dodo

The Twitter OSS Project Builder
Shell
35
star
86

repo-scaffolding

Tools for creating repos based on open source standards and best practices
33
star
87

iago2

A load generator, built for engineers
Scala
24
star
88

caladrius

Performance modelling system for Distributed Stream Processing Systems (DSPS) such as Apache Heron and Apache Storm
Python
22
star
89

ossdecks

Repository for Twitter Open Source Decks
10
star
90

curation-style-guide

Document Repository for Twitter's Curation Style Guide
10
star
91

analytics-infra-governance

Description of the process for how to commit, review, and release code to the Scalding OSS family (Scalding, Summingbird, Algebird, Bijection, Storehaus, etc)
9
star
92

gpl-commitment

Twitter's GPL Cooperation Commitment
5
star
93

second-control-probability-distributions

4
star
94

google-tag-manager-event-tag

Smarty
3
star
95

google-tag-manager-base-tag

Smarty
2
star