UPDATE: We have no plans to continue updating SGT moving forward. Please feel free to clone/fork this project, or try Kolide Fleet as an alternative (https://www.kolide.com/fleet/).
SGT: OSQuery Management Server Built Entirely on AWS!
SGT is an osquery management server written in Golang and built in aws. Sgt (Simple Go TLS) is backed entirely by AWS services, making its infrastructure requirements extremely simple, robust and scalable.
SGT is managed entirely through terraform
NOTE: SGT is under active development. Please help us improve by submitting issues!
Getting started.
NOTE If you are upgrading from a previous version, please see the release notes for 0.2.0
Getting started with sgt is designed to be very simple with minimal setup required. To get started, however, you will need a FEW things first.
TF_WARN_OUTPUT_ERRORS=1
prereqs:
- An AWS account with admin access to DynamoDB, EC2, ES (ElastisearchService), Kinesis/Firehose and IAM. (note, this must be programatic access, so you can have an access key and secret to use)
- Golang 1.9.0+
- Terraform 11.9+
- A domain with DNS managed via Route53 (Note: This does not mean you need to buy a domain, you can use an existing domain and just manage DNS on Route53)
- An SSL cert with public and private keypair. This will be used to terminate TLS connections to our server see Obtaining a free ssl cert for SGT with Letsencrypt for one method of aquiring a certificate
- An aws profile configured.
Installation
-
Clone the repo
git clone [email protected]:OktaSecurityLabs/sgt.git $GOPATH/src/github.com/oktasecuritylabs/sgt -
change into the downloaded directory
cd $GOPATH/src/github.com/oktasecuritylabs/sgt -
Build the project
go build -
Copy your ssl certs to the proper directory. For this example, I'm using a subdomain of example.com with a letsencrypt certificate, sgt-demo.example.com. Lets encrypt certs live in
/etc/letsencrypt/live/<site>so I'm copying them from there into the cert directory for SGT.sudo cp /etc/letsencrypt/live/sgt-demo.example.com/fullchain.pem certs/fullchain.pem sudo cp /etc/letsencrypt/live/sgt-demo.example.com/privkey.pem certs/privkey.pem -
Rename your certs to reflect which site they belong to. I recommend following the example format of
example.domain.com.fullchain.pemmoving...
cd certs mv fullchain.pem sgt-demo.example.com.fullchain.pem mv privkey.pem sgt-demo.example.com.privkey.pem cd .. -
Create a new environment by following the prompts
./sgt wizard6a. Enter a name for your environment (I'm calling my demo one sgt-demo)
Enter new environment name. This is typically something like'Dev' or 'Prod' or 'Testing, but can be anything you want it to be: sgt-demo6b. Choose the AWS profile to use (Mine is again called sgt-demo)
Enter the name for the aws profile you'd like to use to deploy this environment if you've never created a profile before, you can read more about how to do this here http://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html a 'default' profile is created if you've installed and configured the aws cli: sgt-demo6c. Enter the IP address that you are currently deploying from.
Enter an ipaddress or cidr block for access to your elasticsearch cluster. Note: This should probably be your current IP address, as you will need to be able to access elasticsearch via API to create the proper indices and mappings when deploying: xxx.xxx.xxx.xxx/246d. Name your log bucket. I recommend something easily identified for your domain.
Enter a name for the s3 bucket that will hold your osquery logs. Remeber, S3 bucket names must be globally unique: sgt-demo.log.bucket6e. And your config bucket...
Enter a name for the s3 bucket that will hold your server configuration Remember, S3 bucket names must be globally unique: sgt-demo.configuration.bucket6f. Enter your root domain
Enter the domain you will be using for your SGT server. Note: This MUST be a domain which you have previously registered or are managing throughaws. This will be used to create a subdomain for the SGT TLS endpoint example.com6g. Enter the subdomain (sgt-demo in my case)
Enter a subdomain to use as the endpoint. This will be prepended to the domain you provided as a subdomain sgt-demo6h. Enter your aws keypair name
Enter the name of your aws keypair. This is used to access ec2 instances ifthe need should ever arise (it shouldn't). NOTE: This is the name of the keypair EXCLUDING the .pem flie name and it must already exist in aws my-secret-key-name6i. Enter the name of your keypair and priv key, as you named them above.
Enter the name of the full ssl certificate chain bundle you will be using for your SGT server. EG - full_chain.pem : sgt-demo.example.com.fullchain.pem Enter the name of the private key for your ssl certificate. Eg - privkey.pem: sgt-demo.example.com.privkey.pem6j. Enter the node secret
Enter the node secret you will use to enroll your endpoints with the SGT server This secret will be used by each endpoint to authenticate to your server: my-super-secret-node-secret6k. Enter the app secret
Enter the app secret key which will be used to generate session tokens when interacting with the API as an authenticated end-user. Make this long, random and complex: diu3piqeujr302348u33rqwu934r1@#)(*@3Select N when prompted to continue. Because this is a demo environment, we're going to make a small change to our configuration.
-
Edit the environment config file found in
/terraform/<environment/environment.jsonwith your favorite editor and change the value for create_elasticsearch to0. This will disable the creation of elasticsearch, which we will not be using for this demo. In a production environment, Elasticsearch would be a large part of your process, but it adds significant cost and it's not needed for this demo.{ "environment": "example_environment", "aws_profile": "default", "user_ip_address": "127.0.0.1", "sgt_osquery_results_bucket_name": "example_log_bucket_name", "sgt_config_bucket_name": "example_config_bucket_name", "domain": "somedomain.com", "subdomain": "mysubdomain", "aws_keypair": "my_aws_ec2_keypair_name", "full_ssl_certchain": "full_cert_chain.pem", "ssl_private_key": "privkey.pem", "sgt_node_secret": "super_sekret_node_enrollment_key", "sgt_app_secret": "ultra_mega_sekret_key_you'll_never_give_to_anyone_not_even_your_mother", "create_elasticsearch": 0 }
Deploy!!
Its finally time to deploy, although hopefully that wasn't too painful. Deployment is by far the easiest part.
./sgt deploy -env <your environment name> -all
This will stand up the entire environment, including endpoint configuration scripts which we will use to set up some osquery nodes later. The entire process should take about 5-10 minutes depending on your internet connection, at which point you should be ready to install osquery on an endpoint and start receiving logs!
-Note: This getting started guide originally appeared on blog.securelyinsecure.com, but I'm appropriating it for the docs as well, due to it being better than the last readme I wrote.
Once you've installed Go and Terraform, and built your SGT binary, its time to run your deployment!
The wizard will walk you through everything you need to configure a new environment, create the proper directory structure and the environment specific configuration files and stand up the environment if you choose to do so
./sgt wizard
Among other things, the wizard will ask you to provide: The "mail domain for the users of your Kibana dashboard". This should be the domain name used for the email addresses of the people who will be using the Kibana dashboard (example: company.com)
A comma delimited list of users for the Kibana dashboard. The users in the list must correspond to email addresses of the users. For example, if you wanted to initialize Kibana with 2 users (Some Guy, [email protected]; Someone Else, selse@company,com) your input at this prompt woukld be sguy,selse
When you are done with the wizard, you will be prompted to either continue to deploy the actual resources, or exit. If you choose to exit, you you will need manually deploy later
Manual deployment
SGT can be deployed as a full environment, or individual pieces(Note that the components still requires their dependencies to be built, they may just be updated individually to save time)
To deploy SGT...
./sgt deploy -env <environment> -all
To deploy/update an individual component..
./sgt deploy -env <environment> -components elasticsearch,firehose
for a full list of commands, issue the -h flag
If terraform fails at any point during this process, cancel the installation ctrl+c and review
your errors. SGT depends on all previous deploy steps completing successfully, so it is important
to make sure this occurs before moving on to next steps
Creating your first user.
To create a user to interact with SGT, pass the -create_user flag with the requisite options
./sgt create-user -credentials-file <cred_file> -profile <profile> -username <username> -role <"Admin"|"User"|"Read-only">
Getting an Authentication Token
Using any portion of the End-user facing API requires an Authorization token. To get an auth token, send a post request to
/api/v1/get-token supplying your username and password in the post body
{"username": "my_username", "password": "my_password"}If your credentials are valid, you will recieve a json response back
{"Authorization": "<long jtw">}Provide this token in any subsequent requests in the Authorization header
Creating Additional Kibana Users Post-Deployment
- Log into the AWS account where you deployed sgt, and go to the cognito service page
- Click Manage User Pools
- Click the User Pool you created during the sgt deployment
- Click Users and Groups
- Click Create User
- In the Username text box, type the username portion of the user's email address
- Leave the box "Send an invitation to this new user?" checked
- Check the "Email" box
- Un-check the "Mark phone number as verified?" box
- In the Email text box, type the user's email address
- Click Create User
Documentation notes:
Documentation is lacking right now due to a rather un-fun flu season. However, updates to documentation should be expected in teh coming week or so. (This note marked: 1/17/18)
