• Stars
    star
    301
  • Rank 138,451 (Top 3 %)
  • Language
    Go
  • License
    BSD 2-Clause "Sim...
  • Created over 10 years ago
  • Updated over 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Go SSH agent: can execute commands at thousands of servers and upload files to them

GoSSHa: Go SSH agent

Ssh client that supports command execution and file upload on multiple servers (designed to handle thousands of parallel SSH connections). GoSSHa supports SSH authentication using private keys (encrypted keys are supported using external call to ssh-keygen) and ssh-agent, implemented using go.crypto/ssh.

Installation

  1. Install go (programming language) at http://golang.org/
  2. Install GoSSHa: $ go get github.com/YuriyNasretdinov/GoSSHa

Usage

GoSSHa is not designed to be used directly by end users, but rather serve as a lightweight proxy between your application (GUI or CLI) and thousands of SSH connections to remote servers.

Basic protocol

You send commands and receive response by writing and reading JSON lines, for example:

$ GoSSHa
{"Type":"InitializeComplete","InitializeComplete":true}
{"Action":"ssh","Cmd":"uptime","Hosts":["localhost"]}   # your input
{"Type":"ConnectionProgress","ConnectedHost":"localhost"}
{"Type":"Reply","Hostname":"localhost","Stdout":" 1:07  up 1 day,  1:32, 2 users, load averages: 0.90 0.99 1.08\n","Stderr":"","Success":true,"ErrMsg":""}
{"Type":"FinalReply","TotalTime":0.082024023,"TimedOutHosts":{}}

GoSSHa continiously reads stdin and writes response to stdout. The protocol can be split into 2 major phases: initialization and execute loop.

Note: When stdin is closed (EOF), then the program exits even if pending operations are not completed.

Initialization

To be able to run commands GoSSHa examines ~/.ssh/id_rsa, ~/.ssh/id_dsa and ~/.ssh/id_ecdsa if present and asks for their passwords if they are encrypted. If ssh-agent auth socket is present (identified by presence of SSH_AUTH_SOCK environment variable) then it is used as a primary authentication method with fallback to private keys. Password or keyboard-interactive authentication methods are not currently supported, but there are no technical difficulties for adding them.

During initialization, GoSSHa will ask for password for all encrypted private keys it finds, printing message in the following format:

{"Type":"PasswordRequest","PasswordFor":"<path-to-private-key>"}

You can respond with empty object ({}) or provide the passphrase:

{"Password":"<passphrase>"}

In case of any non-critical errors (e.g. you did not provide a passphrase or the passphase is invalid) you will receive message in the following format:

{"Type":"UserError","IsCritical":false,"ErrorMsg":"<error-message>"}

If critical error occurs then all pending operations will be aborted and you will be presented with the same response but "IsCritical":true, for example:

{"Type":"UserError","IsCritical":true,"ErrorMsg":"Cannot parse JSON: unexpected end of JSON input"}

When GoSSHa finishes initialization and is ready to accept commands, the following line will be printed:

{"Type":"InitializeComplete","InitializeComplete":true}

Commands execution

In order to execute a certain <command> on remote servers (e.g. <server1> and <server2>:<port2>):

{"Action":"ssh","Cmd":"<command>","Hosts":["<server1>","<server2>:<port2>"]}

You can also set "Timeout": <timeout> in milliseconds (default is 30000 ms)

While connections to hosts are estabilished and command results are ready you will receive one of the following messages:

  1. Error messages: {"Type":"UserError","IsCritical":false,"ErrorMsg":"<error-message>"}
  2. Connection progress: {"Type":"ConnectionProgress","ConnectedHost":"<hostname>"}
  3. Command result:
{"Type":"Reply","Hostname":"<hostname>","Stdout":"<command-stdout>","Stderr":"<command-stderr>","Success":true|false,"ErrMsg":"<error message>"}

After all commands have done executing or when timeout comes you will receive the following response:

{"Type":"FinalReply","TotalTime":<total-request-time>,"TimedOutHosts":{"<server1>":true,...,"<serverN>":true}}

For your convenience all hosts that timed out are listed in "TimedOutHosts" property, although you could deduce these hosts by subtracting the sets of hostnames that were present in request and the ones present in response.

Note: If you send requests to hosts that previously timed out then GoSSHa may not send {"ConnectedHost":"<hostname>"} for it and only send the command result.

File upload

You can also upload file using the following command:

{"Action":"scp","Source":"<source-file-path>","Target":"<target-file-path>","Hosts":[...]}

You can also set "Timeout": <timeout> in milliseconds (default is 30000 ms) and "MaxThroughput": <max-Bps> in bytes per second to limit maximum bandwidth usage. In default implementation MaxThroughput cannot be less than about 50 Mbit/sec (6553600 bytes per second). If you want to be able to use less bandwidth, try increasing THROUGHPUT_SLEEP_INTERVAL or decreasing MIN_CHUNKS and CHUNK_SIZE constant values in source code.

You will receive progress and results in exactly the same format as for command execution.

Note: Source file contents are fully read in memory, so you should not upload very large files using this command. If you really need to upload huge file to a lot of hosts, try using bittorrent or UFTP, as they provide much higher network effeciency than SSH.

Source code modification

GoSSHa is pretty simple (all it's code is contained in a single file with 500 SLOC) and it should be pretty easy to add new functionality or alter some of it's behaviour. We are always open for pull requests and feature requests as well.

More Repositories

1

chukcha

Easy to use distributed event bus similar to Kafka
Go
93
star
2

social-net

Source codes for youtube "social net in go lang" series
Go
75
star
3

distribkv

Distributed key-value database in Go
Go
40
star
4

WebTerm

Web terminal emulator in various implementations
JavaScript
40
star
5

logscli

CLI for logs based on top of ClickHouse
Go
27
star
6

Dolphin.php

The best file manager in the world
PHP
15
star
7

golang-soft-mocks

Soft Mocks analogue for Golang
Go
13
star
8

chatExample

Example of chat using websockets, for youtube lesson. Do not even try to use it on production!
Go
10
star
9

unrealsync

Utility that can perform bidirectional synchronization between several servers
PHP
8
star
10

hotreload-example

Proof-of-concept for hot code reload in go
Go
6
star
11

hotreload

Hot code reload in Go
Go
5
star
12

clickhouse_compress

Function that implements LZ4 compression in pure go for ClickHouse
Go
4
star
13

go-mysql-admin

MySQL admin panel using websockets and featuring persistent MySQL connection
JavaScript
4
star
14

slightly

Slightly Incorrect Systems YouTube series
Go
4
star
15

instant-im

Demo of network code for really instant messaging that works in bad network conditions
Go
3
star
16

bash

Patched bash to enable colored stderr
C
2
star
17

MEGATROLL.JS

Physics engine my mega trolls
2
star
18

gophdragbench

Benchmark for Gophers&Dragons game at github.com/quasilyte/gophers-and-dragons
Go
2
star
19

tour-generator

Go
1
star
20

datapoliten

Sources for datapoliten.ru
PHP
1
star
21

gotodef

A small experiment to be able to go to definition for go projects in github
1
star
22

gitphp

Performance improvements for GitPHP
PHP
1
star
23

kittenhouse-vs-bulk

Stress-test for kittenhouse and for clickhouse-bulk
Go
1
star
24

yuriynasretdinov.github.io

Random bits and bobs
JavaScript
1
star
25

big

Full replacement for Go math/big package based on GMP library
Go
1
star
26

obsoverlay

OBS overlay for Chukcha that runs tests and displays chat messages from Youtube
Go
1
star