• This repository has been archived on 20/Jul/2020
  • Stars
    star
    261
  • Rank 156,630 (Top 4 %)
  • Language
    Python
  • License
    Other
  • Created almost 10 years ago
  • Updated about 8 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An evented server framework designed for building scalable and introspectable services, built at PayPal.

SuPPort

SuPPort Logo

An evented server framework designed for building scalable and introspectable services and clients based on coroutines, built at PayPal.

How to get SuPPort

pip install SuPPort

You'll also need the following system libraries for the pip install to go through smoothly:

  • libssl-dev
  • libffi-dev

How to use SuPPort

The main entry point into SuPPort is the Group:

import support

server_group = support.Group()

server_group.serve_forever()

Support is completely programmatically configured, so just take a look at the docstring (and examples directory) for more in-depth usage information.

When to use SuPPort

If you need a simple, but tested, preforking webserver to host a WSGI application, use SuPPort. If you need to write servers for non-web protocols, also use SuPPort. If you have a backend service, WSGI or otherwise, SuPPort is much more convenient than setting up Apache/mod_wsgi or other more complex server environments.

Motivation

Greenlet and gevent make a powerful combination, but with power comes complexity. While gevent successfully adds the necessary microthreading management and scheduling fundamentals to greenlet's concurrency primitive, we felt that there was still a significant gap between gevent's functionality and a practical, maintainable server. SuPPort is our design to bridge that gap.

Open-source foundations

SuPPort is built on top of several open-source technologies, so before we dig into the workings of SuPPort, we ought to showcase its foundations:

Design highlights

Even though it represents only a fraction of our Python infrastructure codebase, SuPPort still encompasses too many learnings, big and small, to list here. That said, there are a few aspects of the design worth highlighting. Many motivations have gone into building up a Python stack at PayPal, but as in any enterprise environment, we continuously aim to achieve the following:

Of course organizations of all sizes want these features as well, but the key difference is that large organizations like PayPal usually end up building more. All while demanding a higher degree of redundancy and risk mitigation from their processes. This often results in great cost in terms of both hardware and developer productivity. Fortunately for us, Python can be very efficient in both respects.

So, let's take a stroll through a selection of SuPPort's feature set in the context of these criteria! Note that if you're unfamiliar with evented programming, nonblocking sockets, and gevent in particular, some of this may seem quite foreign. The gevent tutorial is a good entry point for the intermediate Python programmer, which can be supplemented with this well-illustrated introduction to server architectures.

Interoperability

Python usage here at PayPal has spread to virtually every imaginable use case: administrative interfaces, midtier services, operations automation, developer tools, batch jobs; you name it, Python has filled a gap in that area. This legacy has resulted in a few rather interesting abstractions exposed in SuPPort.

BufferedSocket

PayPal has hundreds of services across several tiers. Interoperating between these means having to implement over half a dozen network protocols. The BufferedSocket type eliminated our inevitable code duplication, handling a lot of the nitty-gritty of making a socket into a parser-friendly data source, while retaining timeouts for keeping communications responsive. A must-have primitive for any gevent protocol implementer.

ConnectionManager

Errors happen in live environments. DNS requests fail. Packets are lost. Latency spikes. TCP handshakes are slow. SSL handshakes are slower. Clients rarely handle these problems gracefully. This is why SuPPort includes the ConnectionManager, which provides robust error handling code for all of these cases with consistent logging and monitoring. It also provides a central point of configuration for timeouts and host fallbacks.

Introspectability

As part of a large organization, we can afford to add more machines, and are even required to keep a certain level of redundancy and idle hardware. And while DevOps is catching on in many larger-scale environments, there are many cases in enterprise environments where developers are not allowed to attend to their production code.

SuPPort currently comes with all the same general-purpose introspection capabilities that PayPal Python developers enjoy, meaning that we get you as much structured information about your application as possible without actually requiring login privileges. Of course almost every aspect of this is configurable, to suit a wide variety of environments from development to production.

Context management

Python famously has no global scope: all values are namespaced in module scope. But there are still plenty of aspects of the runtime that are global. Some are out of our control, like the OS-assigned process ID, or the VM-managed garbage collection counters. Other aspects are in our control, and best practice in concurrent programming is to keep these as well-managed as possible.

SuPPort uses a system of Contexts to explicitly manage nonlocal state, eliminating difficult-to-track implicit global state for many core functions. This has the added benefit of creating opportunities to centrally manage and monitor debugging data and statistics, made available through the MetaApplication below.

(Figure 1: see the examples of charts in the the static directory)

MetaApplication

While not exclusively a web server framework, SuPPort leverages its strong roots in the web to provide both a web-based user interface and API full of useful runtime information.

(Figure 2: see the examples of the MetaApplication in the static directory)

As you can see above, there is a lot of information exposed through this default interface. This is partly because of restricted environments not allowing local login on machines, and another part is the relative convenience of a browser for most developers. Not pictured is the feature that the same information is available in JSON format for easy programmatic consumption. Because this application is such a rich source of information, we recommend using SuPPort to run it on a separate port which can be firewalled accordingly, as seen in this example.

Infallibility

At the end of the day, reliability over long periods of time is what earns a stack approval and adoption. At this point, the SuPPort architecture has a billion production requests under its belt here at PayPal, but on the way we put it through the proverbial paces. At various points, we have tested and confirmed these edge behaviors. Here are just a few key characteristics of a well-behaved application:

  • Gracefully sheds traffic under load (no unbounded queues here)
  • Can and has run at 90%+ CPU load for days at a time
  • Is free from framework memory leaks
  • Is robust to memory leakage in user code

To illustrate, a live service handling millions of requests per day had a version of OpenSSL installed which was leaking memory on every handshake. Thanks to preemptive worker cycling on excessive process memory usage, no intervention was required and no customers were impacted. The worker cycling was noted in the logs, the leak was traced to OpenSSL, and operations was notified. The problem was fixed with the next regularly scheduled release rather than being handled as a crisis.

No monkeypatching

One of the first and sometimes only ways that people experience gevent is through monkeypatching. At the top of your main module you issue a call to gevent that automatically swaps out virtually all system libraries with their cooperatively concurrent ones. This sort of magic is relatively rare in Python programming, and rightfully so. Implicit activities like this can have unexpected consequences. SuPPort is a no-monkeypatching approach to gevent. If you want to implement your own network-level code, it is best to use gevent.socket directly. If you want gevent-incompatible libraries to work with gevent, best to use SuPPort's gevent-based threadpooling capabilities, detailed below:

Using threads with gevent

"Threads? In my gevent? I thought the whole point of greenlets and gevent was to eliminate evil, evil threads!" --Countless strawmen

Originating in Stackless and ported over in 2004 by Armin Rigo (of PyPy fame), greenlets are mature and powerful concurrency primitives. We wanted to add that power to the process- and thread-based world of POSIX. There's no point running from standard OS capabilities; threads have their place. Many architectures adopt a thread-per-request or process-per-request model, but the last thing we want is the number of threads going up as load increases. Threads are expensive; each thread adds a bit of contention to the mix, and in many environments the memory overhead alone, typically 4-8MB per thread, presents a problem. At just a few kilobytes apiece, greenlet's microthreads are three orders of magnitude less costly.

Furthermore, thread usage in our architecture is hardly about parallelism; we use worker processes for that. In the SuPPort world, threads are about preemption. Cooperative greenlets are much more efficient overall, but sometimes you really do need guarantees about responsiveness.

One excellent example of how threads provide this responsiveness is the ThreadQueueServer detailed below. But first, there are two built-in Threadpools with decorators worth highlighting, io_bound and cpu_bound:

io_bound

This decorator is primarily used to wrap opaque clients built without affordances for cooperative concurrent IO. We use this to wrap cx_Oracle and other C-based clients that are built for thread-based parallelization. Other major use cases for io_bound is when getting input from standard input (stdin) and files.

(Figure 3: see worker_closeup.png in the static directory)

cpu_bound

The cpu_bound decorator is used to wrap expensive operations that would halt the event loop for too long. We use it to wrap long-running cryptography and serialization tasks, such as decrypting private SSL certificates or loading huge blobs of XML and JSON. Because the majority of use cases' implementations do not release the Global Interpreter Lock, the cpu_bound ThreadPool is actually just a pool of one thread, to minimize CPU contention from multiple unparallelizable CPU-intensive tasks.

It's worth noting that some deserialization tasks are not worth the overhead of dispatching to a separate thread. If the data to be deserialized is very short or a result is already cached. For these cases, we have the cpu_bound_if decorator, which conditionally dispatches to the thread, yielding slightly higher responsiveness for low-complexity requests.

Also note that both of these decorators are reentrant, making dispatch idempotent. If you decorate a function that itself eventually calls a decorated function, performance won't pay the thread dispatch tax twice.

ThreadQueueServer

The ThreadQueueServer exists as an enhanced approach to pulling new connections off of a server's listening socket. It's SuPPort's way of incorporating an industry-standard practice, commonly associated with nginx and Apache, into the gevent WSGI server world.

If you've read this far into the post, you're probably familiar with the standard multi-worker preforking server architecture; a parent process opens a listening socket, forks one or more children that inherit the socket, and the kernel manages which worker gets which incoming client connection.

(Figure 4: see basic_prefork_workers.png in the static directory)

The problem with this approach is that it generally results in inefficient distribution of connections, and can lead to some workers being overloaded while others have cycles to spare. Plus, all worker processes are woken up by the kernel in a race to accept a single inbound connection, in what's commonly referred to as the thundering herd.

The solution implemented here uses a thread that sleeps on accept, removing connections from the kernel's listen queue as soon as possible, then explicitly pushing accepted connections to the main event loop. The ability to inspect this user-space connection queue enables not only even distribution but also intelligent behavior under high load, such as closing incoming connections when the backlog gets too long. This fail-fast approach prevents the kernel from holding open fully-established connections that cannot be reached in a reasonable amount of time. This backpressure takes the wait out of client failure scenarios leading to a more responsive extrinsic system, as well.

What's next for SuPPort

The sections above highlight just a small selection of the features already in SuPPort, and there are many more to cover in future posts. In addition to those, we will also be distilling more code from our internal codebase out into the open. Among these we are particularly excited about:

  • Enhanced human-readable structured logging
  • Advanced network security functionality based on OpenSSL
  • Distributed online statistics collection
  • Additional generalizations for TCP client infrastructure

And of course, more tests! As soon as we get a couple more features distilled out, we'll start porting out more than the skeleton tests we have now. Suffice to say, we're really looking forward to expanding our set of codified concurrent software learnings, and incorporating as much community feedback as possible, so don't forget to watch the repo and subscribe to the blog.

Mahmoud Hashemi, Kurt Rose, Mark Williams, and Chris Lane

Appendix: What SuPPort is (and is not)

  • SuPPort is an architectural supplement to gevent
  • SuPPort is still under active development
  • SuPPort is built for robustness and development speed in the face of a wide range of developer skill levels and environmental complexities
  • SuPPort is not interested in being the next darling of microbenchmarks; it focuses on consistent, robust behavior.
  • SuPPort is one of many effective arrangements of open-source software into a cohesive whole

More Repositories

1

glamorous

DEPRECATED: πŸ’„ Maintainable CSS with React
JavaScript
3,640
star
2

junodb

JunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale.
Go
2,565
star
3

accessible-html5-video-player

Accessible HTML5 Video Player
JavaScript
2,451
star
4

react-engine

a composite render engine for universal (isomorphic) express apps to render both plain react views and react-router views
JavaScript
1,449
star
5

squbs

Akka Streams & Akka HTTP for Large-Scale Production Deployments
Scala
1,433
star
6

PayPal-node-SDK

node.js SDK for PayPal RESTful APIs
JavaScript
1,279
star
7

paypal-checkout-components

please submit Issues about the PayPal JS SDK here: https://github.com/paypal/paypal-js/issues
JavaScript
1,270
star
8

gatt

Gatt is a Go package for building Bluetooth Low Energy peripherals
Go
1,135
star
9

PayPal-iOS-SDK

Accept credit cards and PayPal in your iOS app
Objective-C
974
star
10

gnomon

Utility to annotate console logging statements with timestamps and find slow processes
JavaScript
932
star
11

PayPal-Android-SDK

Accept PayPal and credit cards in your Android app
Java
824
star
12

bootstrap-accessibility-plugin

Accessibility Plugin for Bootstrap 3 and Bootstrap 3 as SubModule
HTML
789
star
13

PayPal-Python-SDK

Python SDK for PayPal RESTful APIs
Python
702
star
14

AATT

Automated Accessibility Testing Tool
JavaScript
601
star
15

PayPal-Ruby-SDK

Ruby SDK for PayPal RESTful APIs
Ruby
593
star
16

ipn-code-samples

PHP
561
star
17

seifnode

C++
545
star
18

PayPal-NET-SDK

.NET SDK for PayPal's RESTful APIs
C#
535
star
19

PayPal-Java-SDK

Java SDK for PayPal RESTful APIs
Java
535
star
20

data-contract-template

Template for a data contract used in a data mesh.
460
star
21

Checkout-PHP-SDK

PHP SDK for Checkout RESTful APIs
PHP
418
star
22

hera

High Efficiency Reliable Access to data stores
Go
289
star
23

SeLion

Enabling Test Automation in Java
Java
281
star
24

nemo-core

Selenium-webdriver based automation in node.js
JavaScript
261
star
25

PayPal-Cordova-Plugin

PayPal SDK Cordova/Phonegap Plugin
Objective-C
248
star
26

gimel

Big Data Processing Framework - Unified Data API or SQL on Any Storage
Scala
245
star
27

scala-style-guide

Style Guidelines for PayPal Scala Applications
240
star
28

merchant-sdk-php

PHP SDK for integrating with PayPal's Express Checkout / MassPay / Web Payments Pro APIs
PHP
230
star
29

paypal-js

Loading wrapper and TypeScript types for the PayPal JS SDK
TypeScript
229
star
30

paypal-rest-api-specifications

This repository contains the specification files for PayPal REST APIs.
192
star
31

resteasy-spring-boot

RESTEasy Spring Boot Starter
Java
188
star
32

Checkout-Java-SDK

PayPal Checkout Java SDK
Java
182
star
33

autosklearn-zeroconf

autosklearn-zeroconf is a fully automated binary classifier. It is based on the AutoML challenge winner auto-sklearn. Give it a dataset with known outcomes (labels) and it returns a list of predicted outcomes for your new data. It even estimates the precision for you! The engine is tuning massively parallel ensemble of machine learning pipelines for best precision/recall.
Python
171
star
34

skipto

SkipTo is a replacement for your old classic "Skipnav" link. Once installed on a site, the script dynamically determines the most important places on the page and presents them to the user in a drop-down menu.
HTML
152
star
35

TLS-update

Documentation & tools for the upcoming TLSv1.2 required update
Java
148
star
36

Checkout-NET-SDK

.NET SDK for Checkout RESTful APIs
C#
139
star
37

cascade

Common Libraries & Patterns for Scala Apps @ PayPal
Scala
129
star
38

merchant-sdk-ruby

Ruby
110
star
39

heap-dump-tool

Tool to sanitize data from Java heap dumps.
Java
110
star
40

NNAnalytics

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.
Java
110
star
41

paypal-smart-payment-buttons

Smart Payment Buttons
JavaScript
108
star
42

yurita

Anomaly detection framework @ PayPal
Scala
106
star
43

InnerSourceCommons

DEPRECATED - old repo for InnerSourceCommons website. Moved to https://github.com/InnerSourceCommons/innersourcecommons.org
JavaScript
105
star
44

adaptivepayments-sdk-php

PHP SDK for integrating with PayPal's AdaptivePayments API
PHP
101
star
45

fullstack-phone

A dual-module phone number system with dynamic regional metadata ☎️
JavaScript
90
star
46

sdk-core-php

for classic PHP SDKs.
PHP
87
star
47

paypal-here-sdk-android-distribution

Add credit card (swipe & key-in) capabilities to your Android app
Java
84
star
48

merchant-sdk-dotnet

C#
83
star
49

paypal-here-sdk-ios-distribution

Add credit card (tap, insert, swipe & key-in) capabilities to your iOS app
Objective-C
82
star
50

payflow-gateway

Repository to store the Payflow Gateway and PayPal Payments Pro SDKs.
C#
80
star
51

sdk-packages

Binary packages for deprecated SDKs.
77
star
52

android-checkout-sdk

Kotlin
77
star
53

Iguanas

Iguanas is a fast, flexible and modular Python package for generating a Rules-Based System (RBS) for binary classification use cases.
Jupyter Notebook
74
star
54

paypal-android

One merchant integration point for all of PayPal's services
Kotlin
72
star
55

legalize.js

JavaScript object validation for browsers + node
JavaScript
70
star
56

paypalcheckout-ios

Need to add Native Checkout to your iOS Application? We can help!
Ruby
70
star
57

paypal-sdk-client

Shared config for PayPal/Braintree client SDKs
JavaScript
65
star
58

load-watcher

Load watcher is a cluster-wide aggregator of metrics, developed for Trimaran: Real Load Aware Scheduler in Kubernetes.
Go
63
star
59

dce-go

Docker Compose Executor to launch pod of docker containers in Apache Mesos.
Go
63
star
60

merchant-sdk-java

Java SDK for integrating with PayPal's Express Checkout / MassPay / Web Payments Pro APIs
Java
62
star
61

sdk-core-java

for classic Java SDKs.
Java
61
star
62

paypal-ios

One merchant integration point for all of PayPal's services
Swift
59
star
63

gorealis

Version 1 of a Go library for interacting with the Aurora Scheduler
Go
58
star
64

scorebot

CSS
57
star
65

PPExtensions

Set of iPython and Jupyter extensions to improve user experience
Python
50
star
66

paypal-checkout-demo

Demo app for paypal-checkout
JavaScript
49
star
67

dione

Dione - a Spark and HDFS indexing library
Scala
49
star
68

Payouts-PHP-SDK

PHP SDK for Payouts RESTful APIs
PHP
49
star
69

pdt-code-samples

Visual Basic
48
star
70

butterfly

Application transformation tool
Java
47
star
71

Payouts-NodeJS-SDK

NodeJS SDK for Payouts RESTful APIs
JavaScript
47
star
72

digraph-parser

Java parser for digraph DSL (Graphviz DOT language)
Java
44
star
73

paypalhttp_php

PHP
43
star
74

tech-talks

Place for all PayPalX presentations, tech talks, and tutorials, and the sample code and apps used in those.
ColdFusion
38
star
75

Illuminator

iOS Automator
Swift
38
star
76

paypal-sdk-release

Unified SDK wrapper module for tests, shared build config, and deploy
JavaScript
37
star
77

PayPal-REST-API-issues

Issue tracking for REST API bugs, features, and documentation requests.
37
star
78

paypal-messaging-components

PayPal JavaScript SDK - messaging components
JavaScript
37
star
79

ionet

ionet is a bridge between the Go stdlib's net and io packages
Go
37
star
80

paypal-access

Examples and code for PayPal Access
Python
36
star
81

horizon

An SBT plugin to help with building, testing, analyzing and releasing Scala
Scala
35
star
82

Payouts-Java-SDK

Java SDK for Payouts RESTful APIs
Java
35
star
83

genio

Genio is an extensible tool that can generate code to consume APIs in multiple programming languages based on different API specification formats.
Ruby
35
star
84

mirakl-hyperwallet-connector

The Hyperwallet Mirakl Connector (HMC) is a self-hosted solution that mediates between a Mirakl marketplace solution and the Hyperwallet (PayPal) payout platform.
Java
34
star
85

openapilint

Node.js linter for OpenAPI specs
JavaScript
31
star
86

paypal-sdk-constants

JavaScript
27
star
87

sdk-core-ruby

Core Library for PayPal Ruby SDKs
Ruby
27
star
88

go.crypto

Go crypto packages
Go
26
star
89

PayPal-PHP-SDK

PHP SDK for PayPal RESTful APIs
PHP
26
star
90

nemo-view

View interface for the Nemo automation framework
JavaScript
26
star
91

Gibberish-Detector-Java

A small program to detect gibberish using a Markov Chain
Java
26
star
92

nemo-accessibility

Automate Accessibility testing within your environment (Localhost)
JavaScript
25
star
93

Payouts-Python-SDK

Python SDK for Payouts RESTful APIs
Python
25
star
94

here-sideloader-api-samples

Sideloader API samples that enable to integrate PayPal Here into other apps
Objective-C
24
star
95

couchbasekafka

Couchbase Kafka Adapter
Java
24
star
96

baler

Bundle assets into iOS static libraries
Python
22
star
97

invoice-sdk-php

PHP SDK for integrating with PayPal's Invoicing API
PHP
21
star
98

Payouts-DotNet-SDK

DotNet SDK for Payouts RESTful APIs
C#
20
star
99

paypal-funding-components

PayPal JavaScript SDK Funding Components
JavaScript
20
star
100

squbs-scala-seed.g8

Scala giter8 Template for Squbs
Scala
20
star