DO .. it! - Framework Toolkit for Sysadmin
DO is a thin framework useful to manage remote servers through ssh.
There are many other alternatives, one of them is capistrano.
So why another one? Basically I need:
- easy creation of my recipes
- see perfectly what's happening on my remote servers
- highly focus on smart actions, upload, download, sudo, replace.
- manage easily more than one server at same time
- a use same syntax to manage local tasks
What really is DO ?
- somewhat like brew
- somewhat like rake
- somewhat like capistrano
All together mixed to make your life easier.
As mentioned before do is a fun mix of capistrano, rake, thor and brew.
With DO you are be able to do easily common task on your local or remote machine. The aim of DO is to become the unique and universal tool language that permit you to make your own brew, apt-get or yum package, same syntax for all your machine.
DO - Installation and Setup
$ sudo gem install do
$ doit download # download a new recipe with --url=
$ doit setup # setup a working home directory
$ doit version # show version number
$ doit list # show task list
$ doit help # show help message
Now you can edit your ~/.do/dorc
adding your servers or plugins.
DO - Files
There are different ways to generate DO tasks, you can:
- Create a file called
Do
orDofile
in your project directory (project wide) - Create
*.rake
files in~/.do
directory (system wide) - Create a file called
dorc
in~/.do
directory (system wide)
You can change the DO home directory (default: ~/.do
) with these
commands:
DO_PATH='/my/new/.do/path'
ENV['DO_PATH']='/my/new/.do/path'
export DO_PATH='/my/new/.do/path'
In this guide we assume that your DO_PATH
is ~/.do
.
DO - Features
- Easily server logging
- SSH connections
- SFTP connections (upload and download)
- run cmd (we handle also with input)
- shortcuts (exist?, read, replace, append etc...)
Code is much better:
server = DO::Server.new('srv1', 'srv1.domain.local', 'root', :key => %w[srv1.pem]
server.run 'uname'
# root@srv1 ~ # uname
# Linux
server.run 'uname', '-a'
# root@srv1 ~ # uname -a
# Linux srv1.lipsiasoft.net 2.6.18-194.32.1.el5 x86_64 x86_64 x86_64 GNU/Linux
server.run 'mysqladmin -u root -p password "oldone"', 'newpassword'
# root@srv1 ~ # mysqladmin -u root -p password 'oldone'
# Enter password: oldone
# mysqladmin: connect to server at 'localhost' failed
# error: 'Access denied for user 'root'@'localhost' (using password: YES)'
server.exist?('~/.ssh')
# root@srv1 ~ # test -e ~/.ssh && echo True
# => true
server.read('/etc/redhat-release')
# root@srv1 ~ # cat /etc/redhat-release
# => "CentOS release 5.5 (Final)"
server.upload '/tmp/file', '/tmp/foo'
# root@srv1 ~ # upload from '/tmp/file' to '/tmp/foo'
server.download '/tmp/foo', '/tmp/file2'
# root@srv1 ~ # download from '/tmp/foo' to '/tmp/file2'
server.replace :all, 'new content', '/tmp/file'
# root@srv1 ~ # replace all in '/tmp/foo'
server.read('/tmp/foo')
# root@srv1 ~ # cat /tmp/foo
# => "new content"
server.replace /content$/, 'changed content', '/tmp/foo'
# root@srv1 ~ # replace /content$/ in '/tmp/foo'
server.read('/tmp/foo')
# root@srv1 ~ # cat /tmp/foo
# => "new changed content"
server.append('appended', '/tmp/foo')
# root@srv1 ~ # append to 'bottom' in '/tmp/foo'
server.read('/tmp/foo')
# root@srv1 ~ # cat /tmp/foo
# => "new changed contentappended"
server.append('---', '/tmp/foo', :top)
# root@srv1 ~ # append to 'top' in '/tmp/foo'
server.read('/tmp/foo')
# root@srv1 ~ # cat /tmp/foo
# => "---new changed contentappended"
server.ask "Please choose"
# root@srv1 ~ # Please choose: foo
# => "foo"
server.yes? "Do you want to proceed"
# root@srv1 ~ # Do you want to proceed? (y/n): y
# => 0
server.wait
# Press ENTER to continue...
DO - Plugins
DO, support plugins, you can manually add new one in your ~/.do/dorc
with a simple line:
plugin :vim, 'https://raw.github.com/DAddYE/.do/master/vim.rake'
However we have a doit
command for that:
$ doit download --url https://raw.github.com/DAddYE/.do/master/vim.rake
This command add for you a new line in your ~/.do/dorc
and perform:
$ doit plugin:vim
which download and install in your ~/.do/dorc
directory a new rake
file.
Once this happen you are be able to see new tasks:
$ doit list
doit plugin:vim # install vim plugin
doit setup # setup a working home directory
doit vim:configure # configure with a janus custom template
doit vim:install # install vim with python and ruby support
Scenario and examples
I'm porting my custom recipes to DO, you can find my dot files here.
Here servers definitions:
# ~/.do/dorc
keys = %w(/keys/master.pem /keys/instances.pem /keys/stage.pem)
server :sho0, 'sho0.lipsiasoft.biz', 'root', :keys => keys
server :srv0, 'srv0.lipsiasoft.biz', 'root', :keys => keys
server :srv1, 'srv1.lipsiasoft.biz', 'root', :keys => keys
server :srv2, 'srv2.lipsiasoft.biz', 'root', :keys => keys
I've also some recipes in my ~/.do
path:
# ~/.do/configure.rake
namespace :configure
desc "upgrade rubygems and install useful gems"
task :gems => :ree, :in => :web do
run "gem update --system" if yes?("Do you want to update rubygems?")
run "gem install rake"
run "gem install highline"
run "gem install bundler"
run "ln -s /opt/ruby-enterprise/bin/bundle /usr/bin/bundle" unless exist?("/usr/bin/bundle")
end
desc "create motd for each server"
task :motd, :in => :remote do
replace :all, "Hey boss! Welcome to the \e[1m#{name}\e[0m of LipsiaSOFT s.r.l.\n", "/etc/motd"
end
desc "redirect emails to a real account"
task :root_emails, :in => :remote do
append "\nroot: [email protected]", "/etc/aliases"
run "newaliases"
end
desc "mysql basic configuration"
task :mysql => :yum, :in => :remote do
run "yum install mysql-server mysql mysql-devel -y"
run "chkconfig --level 2345 mysqld on"
run "service mysqld restart"
pwd = ask "Tell me the password for mysql"
run "mysqladmin -u root -p password '#{pwd}'", :input => "\n"
run "service mysqld restart"
run "ln -fs /var/lib/mysql/mysql.sock /tmp/mysql.sock"
end
...
I call these task with:
$ doit configure:gems
$ doit configure:motd
$ doit configure:mysql
NOTE like rake tasks you are be able to add prerequisites to any task, ex:
task :mysql => :yum, :in => :remote do; ...; end
That's are some local tasks:
namespace :l do
task :setup do
name = File.basename(File.expand_path('.'))
exit unless yes?('Do you want to setup "%s"?' % name)
srv = nil
until servers.map(&:name).include?(srv)
srv = ask("Which server do you want to use (%s)" % servers.map(&:name).join(", ")).to_sym
end
if File.exist?('.git')
exit unless yes?('Project "%s" has already a working repo, do you want remove it?' % name)
run 'rm -rf .git'
end
run 'git init'
run 'git remote add origin [email protected]:/%s.git' % name
run_task('l:commit') if yes?("Are you ready to commit it, database, config etc is correct?")
end
task :commit do
run 'git add .'
run 'git commit -a'
run 'git push origin master'
end
end
When I need to setup a new project:
$ doit l:setup
$ doit l:commit # to make a fast commit and push
As you can see, define remote and local task, is simple like making
standard rake tasks. DO extend Rake
.
DO - Filtering
Sometimes you want to perform a task only on some servers:
$ doit configure:new --srv1 # apply recipes only on srv1
$ doit configure:new --no-srv1 # apply recipes to all except srv1
DO - Output (Awesome...)
What I really wanted was a great understandable, colored output that clarify me what's happen on my remote servers.
Here a example output of my tasks:
root@sho0 ~ # touch /root/.bashrc
root@sho0 ~ # replace all in '/root/.bashrc'
root@sho0 ~ # replace all in '/etc/motd'
root@sho0 ~ # replace /HOSTNAME=.*/ in '/etc/sysconfig/network'
root@sho0 ~ # chmod +w /etc/sudoers
root@sho0 ~ # replace /^Defaults requiretty/ in '/etc/sudoers'
root@sho0 ~ # chkconfig --level 2345 sendmail on
root@sho0 ~ # chkconfig --level 2345 mysqld on
root@sho0 ~ # service sendmail restart
Shutting down sm-client: [ OK ]
Shutting down sendmail: [ OK ]
Starting sendmail: [ OK ]
Starting sm-client: [ OK ]
root@sho0 ~ # service mysqld restart
Stopping mysqld: [ OK ]
Starting mysqld: [ OK ]
root@sho0 ~ # Tell me the password for mysql: xx
root@sho0 ~ # mysqladmin -u root -p password xx'
Enter password: xx
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
root@sho0 ~ # service mysqld restart
Stopping mysqld: [ OK ]
Starting mysqld: [ OK ]
root@sho0 ~ # ln -fs /var/lib/mysql/mysql.sock /tmp/mysql.sock
Copyright
Copyright (C) 2011 Davide D'Agostino - @daddye
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the โSoftwareโ), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED โAS ISโ, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.