• Stars
    star
    300
  • Rank 138,870 (Top 3 %)
  • Language
    Ruby
  • License
    MIT License
  • Created almost 10 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Module for the 'fog' gem to support Amazon Web Services http://aws.amazon.com/

Fog::Aws

Gem Version Build Status Test Coverage Code Climate

Installation

Add this line to your application's Gemfile:

gem 'fog-aws'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fog-aws

Usage

Before you can use fog-aws, you must require it in your application:

require 'fog/aws'

Since it's a bad practice to have your credentials in source code, you should load them from default fog configuration file: ~/.fog. This file could look like this:

default:
  aws_access_key_id:     <YOUR_ACCESS_KEY_ID>
  aws_secret_access_key: <YOUR_SECRET_ACCESS_KEY>

EC2

Connecting to the EC2 Service:

ec2 = Fog::Compute.new :provider => 'AWS', :region => 'us-west-2'

You can review all the requests available with this service using #requests method:

ec2.requests # => [:allocate_address, :assign_private_ip_addresses, :associate_address, ...]

Launch an EC2 on-demand instance:

response = ec2.run_instances(
  "ami-23ebb513",
  1,
  1,
  "InstanceType"  => "t1.micro",
  "SecurityGroup" => "ssh",
  "KeyName"       => "miguel"
)
instance_id = response.body["instancesSet"].first["instanceId"] # => "i-02db5af4"
instance = ec2.servers.get(instance_id)
instance.wait_for { ready? }
puts instance.public_ip_address # => "356.300.501.20"

Terminate an EC2 instance:

instance = ec2.servers.get("i-02db5af4")
instance.destroy

Fog::AWS is more than EC2 since it supports many services provided by AWS. The best way to learn and to know about how many services are supported is to take a look at the source code. To review the tests directory and to play with the library in bin/console can be very helpful resources as well.

S3

Connecting to the S3 Service:

s3 = Fog::Storage.new(provider: 'AWS', region: 'eu-central-1')

Creating a file:

directory = s3.directories.new(key: 'gaudi-portal-dev')
file = directory.files.create(key: 'user/1/Gemfile', body: File.open('Gemfile'), tags: 'Org-Id=1&Service-Name=My-Service')

Listing files:

directory = s3.directories.get('gaudi-portal-dev', prefix: 'user/1/')
directory.files

Warning! s3.directories.get retrieves and caches meta data for the first 10,000 objects in the bucket, which can be very expensive. When possible use s3.directories.new.

Generating a URL for a file:

directory.files.new(key: 'user/1/Gemfile').url(Time.now + 60)
Generate download URL

You should pass an option argument that contains the query key with response-content-disposition inside indicating that is an attachment and the filename to be used when downloaded.

options = {
  query: {
    'response-content-disposition' => "attachment; filename=#{key}"
  }
}

directory.files.new(key: 'user/1/Gemfile').url(Time.now + 60, options)
Controlling credential refresh time with IAM authentication

When using IAM authentication with temporary security credentials, generated S3 pre-signed URLs only last as long as the temporary credential.

Generating the URLs in the following manner will return a URL that will not last as long as its requested expiration time if the remainder of the authentication token lifetime was shorter.

s3 = Fog::Storage.new(provider: 'AWS', use_iam_profile: true)
directory = s3.directories.get('gaudi-portal-dev', prefix: 'user/1/')

directory.files.new(key: 'user/1/Gemfile').url(Time.now + 60)

By default the temporary credentials in use are refreshed only within the last 15 seconds of its expiration time. The URL requested with 60 seconds lifetime using the above example will only remain valid for 15 seconds in the worst case.

The problem can be avoided by refreshing the token early and often, by setting configuration aws_credentials_refresh_threshold_seconds (default: 15) which controls the time when the refresh must occur. It is expressed in seconds before the temporary credential's expiration time.

The following example can ensure pre-signed URLs last as long as 60 seconds by automatically refreshing the credentials when its remainder lifetime is lower than 60 seconds:

s3 = Fog::Storage.new(
  provider: 'AWS',
  use_iam_profile: true,
  aws_credentials_refresh_threshold_seconds: 60
)
directory = s3.directories.get('gaudi-portal-dev', prefix: 'user/1/')

directory.files.new(key: 'user/1/Gemfile').url(Time.now + 60)

Copying a file

directory = s3.directories.new(key: 'gaudi-portal-dev')
file = directory.files.get('user/1/Gemfile')
file.copy("target-bucket", "user/2/Gemfile.copy")

To speed transfers of large files, the concurrency option can be used to spawn multiple threads. Note that the file must be at least 5 MB for multipart uploads to work. For example:

directory = s3.directories.new(key: 'gaudi-portal-dev')
file = directory.files.get('user/1/Gemfile')
file.multipart_chunk_size = 10 * 1024 * 1024
file.concurrency = 10
file.copy("target-bucket", "user/2/Gemfile.copy")

Documentation

See the online documentation for a complete API reference.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

Contributing

  1. Fork it ( https://github.com/fog/fog-aws/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

More Repositories

1

fog

The Ruby cloud services library.
Ruby
4,318
star
2

fog-google

Fog for Google Cloud Platform
Ruby
98
star
3

fog-openstack

Fog for OpenStack Platform
Ruby
68
star
4

fog-core

fog's core, shared behaviors without API and provider specifics
Ruby
45
star
5

fog-aliyun

Fog provider for aliyun
Ruby
37
star
6

fog-vsphere

Fog for vSphere
Ruby
36
star
7

fog-azure-rm

Fog for Azure Resource Manager
Ruby
35
star
8

fog-proxmox

Fog module for Proxmox VE Platform
Ruby
31
star
9

fog-backblaze

Integration library for gem fog and Backblaze B2 Cloud Storage
Ruby
19
star
10

fog-azure

Ruby
19
star
11

fog-digitalocean

Fog for DigitalOcean Platform — Edit
Ruby
17
star
12

fog-local

Module for the 'fog' gem to support local filesystem storage
Ruby
17
star
13

fog-libvirt

libvirt provider for fog
Ruby
16
star
14

fog-xenserver

Module for the 'fog' gem to support XENSERVER
Ruby
16
star
15

fog-sakuracloud

Module for the 'fog' gem to support Sakura no Cloud
Ruby
13
star
16

fog-dropbox

Module for the 'fog' gem to support Dropbox http://dropbox.com
Ruby
12
star
17

fog-ovirt

Ruby
11
star
18

fog-rackspace

Rackspace provider gem for Fog ecosystem
Ruby
8
star
19

fog.github.com

fog.io documentation website
JavaScript
8
star
20

fog-dnsimple

Module for the 'fog' gem to support DNSimple.
Ruby
7
star
21

fog-openstack-core

fog's core openstack behaviors without API and cloud provider specifics
Ruby
6
star
22

fog-brightbox

Brightbox Cloud support for the fog gem. Officially supported.
Ruby
5
star
23

fog-radosgw

Ruby
5
star
24

fog-oraclecloud

Fog library for the Oracle Cloud
Ruby
5
star
25

fog-kubevirt

Module for the 'fog' gem to support Kubevirt.
Ruby
5
star
26

fog-powerdns

Module for the 'fog' gem to support PowerDNS
Ruby
4
star
27

fog-internet-archive

Internet Archive Storage support for Fog
Ruby
4
star
28

fog-akamai

Module for the 'fog' gem to support Akamai http://www.akamai.com/
Ruby
3
star
29

fog-profitbricks

Module for the 'fog' gem to support ProfitBricks
Ruby
3
star
30

fog-dynect

Module for the 'fog' gem to support Dyn Managed DNS http://dyn.com/
Ruby
3
star
31

fog-xml

Shared XML related functionality for fog
Ruby
2
star
32

fog-hadoop

Fog connector for Hadoop
Ruby
2
star
33

fog-cloudatcost

Module for the 'fog' gem to support CloudAtCost Services http://panel.cloudatcost.com/
Ruby
2
star
34

fog-linode

Fog provider for Linode
Ruby
2
star
35

fog-riakcs

Module for the 'fog' gem to support RiakCS
Ruby
2
star
36

fog-json

Shared JSON related functionality for fog
Ruby
2
star
37

fog-packet

Fog for Packet
Ruby
2
star
38

fog-cloudstack

Ruby
1
star
39

fog-joyent

Module for the 'fog' gem to support Joyent
Ruby
1
star
40

fog-softlayer

Ruby
1
star
41

fog-ecloud

Module for the 'fog' gem to support Terremark Enterprise Cloud
Ruby
1
star
42

fog-vmfusion

Module for the 'fog' gem to support VMWARE Fusion
Ruby
1
star
43

fog-dnsmadeeasy

Fog DNS Made Easy Module
Ruby
1
star