• Stars
    star
    289
  • Rank 143,419 (Top 3 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

High Efficiency Reliable Access to data stores

Go Build Java CI with Maven License

Hera - High Efficiency Reliable Access to data stores

Hera multiplexes connections for MySQL, Oracle and PostgreSQL databases. It supports sharding the databases for horizontal scaling.

What is Hera

Hera is Data Access Gateway that helps to enable scaling and improving the availability of databases.

  • Protects the database from resource exhaustion by evicting poorly performing queries
  • Intelligently routes read/write traffic appropriately for better load balancing
  • Improves tolerance to database outages
  • Provides high performance secured connections between applications and Hera
  • Provides domain agnostic database sharding for horizontal database scaling
  • Automatic transaction application failover between replica databases
  • And many more site resiliency features

Getting Started

You can build mux using either Docker or manual build.

Docker Build

For development, the following docker commands for the appropriate environment can help get started

Linux

git clone https://github.com/paypal/hera.git
docker run --network host --name mysql-11 -e MYSQL_ROOT_PASSWORD=62-AntHill -d mysql:latest
cd hera/tests/devdocker
docker exec -i mysql-11 mysql -u root -h 127.0.0.1 -p62-AntHill -t < sample.sql
mkdir -p src/github.com/paypal/hera
rsync -av --exclude tests/devdocker ../.. src/github.com/paypal/hera
docker build -t hera-oss .
docker run -it --rm --name testRunHeraOss --network host -e password=62-AntHill hera-oss

Mac

git clone https://github.com/paypal/hera.git
docker network create my-network
docker run --network my-network --name mysql-11 -e MYSQL_ROOT_PASSWORD=62-AntHill -d mysql:latest
cd hera/tests/devdocker
docker exec -i mysql-11 mysql -u root -h 127.0.0.1 -p62-AntHill -t < sample.sql
mkdir -p src/github.com/paypal/hera
rsync -av --exclude tests/devdocker ../.. src/github.com/paypal/hera
sed -i.bak -e 's/127.0.0.1/mysql-11/g' srv/start.sh
docker build -t hera-oss .
docker run -it --rm --name testRunHeraOss --network my-network -p 10101:10101 -e password=62-AntHill hera-oss

To test it, in a separate terminal:

docker exec -it testRunHeraOss /bin/bash
cd /go/src
go run github.com/paypal/hera/client/gosqldriver/tls/example/sample_main.go

Manual Build

The following sections explain the process for manually building mux without Docker. We only tested on RedHat and Ubuntu.

Install Dependencies

  1. Install Go 1.10+.
  2. Install MySQL, Oracle or PostgresSQL.
  3. Install the MySQL driver, Oracle driver and the PostgreSQL driver
  4. Install Oracle instant client.

Build Binaries

  1. Navigate to the working directory.

    cd $WORKSPACE
    export GOPATH=$WORKSPACE
    
  2. Option 1

    Clone the source code from github

    git clone [email protected]:paypal/hera src/github.com/paypal/hera
    

    Option 2

    (a) GO 1.12 is prerequisite

    (b) export GO111MODULE=on ( to enable the go mod feature)

    go get github.com/paypal/hera
    
  3. Install the dependencies: MySQL, Oracle and PostgreSQL driver

    go get github.com/go-sql-driver/mysql
    go get github.com/godror/godror
    go get github.com/lib/pq
    
  4. Build server binaries

    go install github.com/paypal/hera/mux github.com/paypal/hera/worker/mysqlworker github.com/paypal/hera/worker/oracleworker github.com/paypal/hera/worker/postgresworker
    
  5. Build Go test client

    go install github.com/paypal/hera/tests/e2e/client
    
  6. Build Java test client under the client/java directory

  7. Build the C++ oracleworker

    cd worker/cppworker/worker
    make -f ../build/makefile19
    

Running the server

To run mux there is minimal configuration required. Please see examples for running with MySQL or Oracle. The main configuration file is hera.txt, which must contain the TCP port where the server listens and the number of workers. The user name, password and the data source are passed via environment parameters.

    # the proxy executable
    ln -s $GOPATH/bin/mux .
    # the MySQL worker
    ln -s $GOPATH/bin/mysqlworker mysqlworker
    # to use the Oracle worker use oracleworker instead of mysqlworker

    # create the configuration file with the required configuration
    echo 'bind_ip=127.0.0.1' > hera.txt
    echo 'bind_port=11111' >> hera.txt
    echo 'opscfg.hera.server.max_connections=2' >> hera.txt
    # if using mysql run this
    # echo 'database_type=mysql' >> hera.txt
    # if using postgres run this
    # echo 'database_type=postgres' >> hera.txt

    # create minimal CAL configuration, needed by ops config
    echo 'cal_pool_name=stage_hera' > cal_client.txt

    # the database user name
    export username='user'
    # the database password
    export password='pass'
    # the data source
    export TWO_TASK='tcp(127.0.0.1:3306)/myschema'
    # for Oracle the datasource can be like '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=hostname)
    #    (PORT=12345)))(CONNECT_DATA=(SERVICE_NAME=sn)))'.
    # for Oracle only add to LD_LIBRARY_PATH environment variable the path to the shared libraries of the
    #    Oracle instant client
    # for postgress format can be like : export TWO_TASK='127.0.0.1:5432/user?connect_timeout=60'

    # start
    ./mux --name hera-test
    
    # to validate HERA is running fine tail 'state-log' file and check 2 connections in 'acpt' state.
    # 01/07/2022 18:28:39: -----------  init  acpt  wait  busy  schd  fnsh  quce  asgn  idle  bklg  strd
    # 01/07/2022 18:28:39: hera            0     2     0     0     0     0     0     0     0     0     0
    
    # incase connections are not in accept state, check the hera.log file for errors

For details about the parameters see configuration

Running the client

There is a Java client implemented as JDBC driver. Please see the documentation for how to install and use it.

There is also a Go client implemented as SQL driver. Please see the example.

License

Hera is licensed under Apache 2.0.

More Repositories

1

glamorous

DEPRECATED: 💄 Maintainable CSS with React
JavaScript
3,640
star
2

junodb

JunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale.
Go
2,565
star
3

accessible-html5-video-player

Accessible HTML5 Video Player
JavaScript
2,451
star
4

react-engine

a composite render engine for universal (isomorphic) express apps to render both plain react views and react-router views
JavaScript
1,449
star
5

squbs

Akka Streams & Akka HTTP for Large-Scale Production Deployments
Scala
1,433
star
6

PayPal-node-SDK

node.js SDK for PayPal RESTful APIs
JavaScript
1,279
star
7

paypal-checkout-components

please submit Issues about the PayPal JS SDK here: https://github.com/paypal/paypal-js/issues
JavaScript
1,270
star
8

gatt

Gatt is a Go package for building Bluetooth Low Energy peripherals
Go
1,135
star
9

PayPal-iOS-SDK

Accept credit cards and PayPal in your iOS app
Objective-C
974
star
10

gnomon

Utility to annotate console logging statements with timestamps and find slow processes
JavaScript
932
star
11

PayPal-Android-SDK

Accept PayPal and credit cards in your Android app
Java
824
star
12

bootstrap-accessibility-plugin

Accessibility Plugin for Bootstrap 3 and Bootstrap 3 as SubModule
HTML
789
star
13

PayPal-Python-SDK

Python SDK for PayPal RESTful APIs
Python
702
star
14

AATT

Automated Accessibility Testing Tool
JavaScript
601
star
15

PayPal-Ruby-SDK

Ruby SDK for PayPal RESTful APIs
Ruby
593
star
16

ipn-code-samples

PHP
561
star
17

seifnode

C++
545
star
18

PayPal-NET-SDK

.NET SDK for PayPal's RESTful APIs
C#
535
star
19

PayPal-Java-SDK

Java SDK for PayPal RESTful APIs
Java
535
star
20

data-contract-template

Template for a data contract used in a data mesh.
460
star
21

Checkout-PHP-SDK

PHP SDK for Checkout RESTful APIs
PHP
418
star
22

SeLion

Enabling Test Automation in Java
Java
281
star
23

nemo-core

Selenium-webdriver based automation in node.js
JavaScript
261
star
24

support

An evented server framework designed for building scalable and introspectable services, built at PayPal.
Python
261
star
25

PayPal-Cordova-Plugin

PayPal SDK Cordova/Phonegap Plugin
Objective-C
248
star
26

gimel

Big Data Processing Framework - Unified Data API or SQL on Any Storage
Scala
245
star
27

scala-style-guide

Style Guidelines for PayPal Scala Applications
240
star
28

merchant-sdk-php

PHP SDK for integrating with PayPal's Express Checkout / MassPay / Web Payments Pro APIs
PHP
230
star
29

paypal-js

Loading wrapper and TypeScript types for the PayPal JS SDK
TypeScript
229
star
30

paypal-rest-api-specifications

This repository contains the specification files for PayPal REST APIs.
192
star
31

resteasy-spring-boot

RESTEasy Spring Boot Starter
Java
188
star
32

Checkout-Java-SDK

PayPal Checkout Java SDK
Java
182
star
33

autosklearn-zeroconf

autosklearn-zeroconf is a fully automated binary classifier. It is based on the AutoML challenge winner auto-sklearn. Give it a dataset with known outcomes (labels) and it returns a list of predicted outcomes for your new data. It even estimates the precision for you! The engine is tuning massively parallel ensemble of machine learning pipelines for best precision/recall.
Python
171
star
34

skipto

SkipTo is a replacement for your old classic "Skipnav" link. Once installed on a site, the script dynamically determines the most important places on the page and presents them to the user in a drop-down menu.
HTML
152
star
35

TLS-update

Documentation & tools for the upcoming TLSv1.2 required update
Java
148
star
36

Checkout-NET-SDK

.NET SDK for Checkout RESTful APIs
C#
139
star
37

cascade

Common Libraries & Patterns for Scala Apps @ PayPal
Scala
129
star
38

merchant-sdk-ruby

Ruby
110
star
39

heap-dump-tool

Tool to sanitize data from Java heap dumps.
Java
110
star
40

NNAnalytics

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.
Java
110
star
41

paypal-smart-payment-buttons

Smart Payment Buttons
JavaScript
108
star
42

yurita

Anomaly detection framework @ PayPal
Scala
106
star
43

InnerSourceCommons

DEPRECATED - old repo for InnerSourceCommons website. Moved to https://github.com/InnerSourceCommons/innersourcecommons.org
JavaScript
105
star
44

adaptivepayments-sdk-php

PHP SDK for integrating with PayPal's AdaptivePayments API
PHP
101
star
45

fullstack-phone

A dual-module phone number system with dynamic regional metadata ☎️
JavaScript
90
star
46

sdk-core-php

for classic PHP SDKs.
PHP
87
star
47

paypal-here-sdk-android-distribution

Add credit card (swipe & key-in) capabilities to your Android app
Java
84
star
48

merchant-sdk-dotnet

C#
83
star
49

paypal-here-sdk-ios-distribution

Add credit card (tap, insert, swipe & key-in) capabilities to your iOS app
Objective-C
82
star
50

payflow-gateway

Repository to store the Payflow Gateway and PayPal Payments Pro SDKs.
C#
80
star
51

sdk-packages

Binary packages for deprecated SDKs.
77
star
52

android-checkout-sdk

Kotlin
77
star
53

Iguanas

Iguanas is a fast, flexible and modular Python package for generating a Rules-Based System (RBS) for binary classification use cases.
Jupyter Notebook
74
star
54

paypal-android

One merchant integration point for all of PayPal's services
Kotlin
72
star
55

legalize.js

JavaScript object validation for browsers + node
JavaScript
70
star
56

paypalcheckout-ios

Need to add Native Checkout to your iOS Application? We can help!
Ruby
70
star
57

paypal-sdk-client

Shared config for PayPal/Braintree client SDKs
JavaScript
65
star
58

load-watcher

Load watcher is a cluster-wide aggregator of metrics, developed for Trimaran: Real Load Aware Scheduler in Kubernetes.
Go
63
star
59

dce-go

Docker Compose Executor to launch pod of docker containers in Apache Mesos.
Go
63
star
60

merchant-sdk-java

Java SDK for integrating with PayPal's Express Checkout / MassPay / Web Payments Pro APIs
Java
62
star
61

sdk-core-java

for classic Java SDKs.
Java
61
star
62

paypal-ios

One merchant integration point for all of PayPal's services
Swift
59
star
63

gorealis

Version 1 of a Go library for interacting with the Aurora Scheduler
Go
58
star
64

scorebot

CSS
57
star
65

PPExtensions

Set of iPython and Jupyter extensions to improve user experience
Python
50
star
66

paypal-checkout-demo

Demo app for paypal-checkout
JavaScript
49
star
67

dione

Dione - a Spark and HDFS indexing library
Scala
49
star
68

Payouts-PHP-SDK

PHP SDK for Payouts RESTful APIs
PHP
49
star
69

pdt-code-samples

Visual Basic
48
star
70

butterfly

Application transformation tool
Java
47
star
71

Payouts-NodeJS-SDK

NodeJS SDK for Payouts RESTful APIs
JavaScript
47
star
72

digraph-parser

Java parser for digraph DSL (Graphviz DOT language)
Java
44
star
73

paypalhttp_php

PHP
43
star
74

tech-talks

Place for all PayPalX presentations, tech talks, and tutorials, and the sample code and apps used in those.
ColdFusion
38
star
75

Illuminator

iOS Automator
Swift
38
star
76

paypal-sdk-release

Unified SDK wrapper module for tests, shared build config, and deploy
JavaScript
37
star
77

PayPal-REST-API-issues

Issue tracking for REST API bugs, features, and documentation requests.
37
star
78

paypal-messaging-components

PayPal JavaScript SDK - messaging components
JavaScript
37
star
79

ionet

ionet is a bridge between the Go stdlib's net and io packages
Go
37
star
80

paypal-access

Examples and code for PayPal Access
Python
36
star
81

horizon

An SBT plugin to help with building, testing, analyzing and releasing Scala
Scala
35
star
82

Payouts-Java-SDK

Java SDK for Payouts RESTful APIs
Java
35
star
83

genio

Genio is an extensible tool that can generate code to consume APIs in multiple programming languages based on different API specification formats.
Ruby
35
star
84

mirakl-hyperwallet-connector

The Hyperwallet Mirakl Connector (HMC) is a self-hosted solution that mediates between a Mirakl marketplace solution and the Hyperwallet (PayPal) payout platform.
Java
34
star
85

openapilint

Node.js linter for OpenAPI specs
JavaScript
31
star
86

paypal-sdk-constants

JavaScript
27
star
87

sdk-core-ruby

Core Library for PayPal Ruby SDKs
Ruby
27
star
88

go.crypto

Go crypto packages
Go
26
star
89

PayPal-PHP-SDK

PHP SDK for PayPal RESTful APIs
PHP
26
star
90

nemo-view

View interface for the Nemo automation framework
JavaScript
26
star
91

Gibberish-Detector-Java

A small program to detect gibberish using a Markov Chain
Java
26
star
92

nemo-accessibility

Automate Accessibility testing within your environment (Localhost)
JavaScript
25
star
93

Payouts-Python-SDK

Python SDK for Payouts RESTful APIs
Python
25
star
94

here-sideloader-api-samples

Sideloader API samples that enable to integrate PayPal Here into other apps
Objective-C
24
star
95

couchbasekafka

Couchbase Kafka Adapter
Java
24
star
96

baler

Bundle assets into iOS static libraries
Python
22
star
97

invoice-sdk-php

PHP SDK for integrating with PayPal's Invoicing API
PHP
21
star
98

Payouts-DotNet-SDK

DotNet SDK for Payouts RESTful APIs
C#
20
star
99

paypal-funding-components

PayPal JavaScript SDK Funding Components
JavaScript
20
star
100

squbs-scala-seed.g8

Scala giter8 Template for Squbs
Scala
20
star