Exploring GateOne Browser SSH terminal

I came across a program called Gate One by LiftOff Software that
just amazed me. This is an open-source, web-based ssh terminal. It is
capable of multiple users, sessions, and bookmarks. I’ve tried a number
of AJAX terminals or Java applet based ones in the past. The javascript
ones usually did not have very good terminal emulation, while the Java
apps worked, but worked just like a local desktop app (making it’s own
connection to port 22). Gate One uses WebSockets, allowing for full
duplex communication through your web browser over the same port 80 or
443 used to serve up the web page.

Installation

Gate One is a python application using the Tornado framework. As
such, at runs independently of an existing web server and handles
connections from browsers internally. If you already have a web server
running on your system, you will need to tell Gate One to use a
different IP or a different port.

Installation using pre-built binaries or the source is fairly
straightforward and detailed in the documentation.

The installer creates a directory of /opt/gateone and places all
necessary files there. You can run it by changing to that directory and
running gateone.py as root.

[email protected]:/opt/gateone$ sudo ./gateone.py
[W 120801 13:52:06 terminal:166] Could not import the Python Imaging Library (PIL) so images will not be displayed in the terminal
[W 120801 13:52:06 gateone:2232] dtach command not found.  dtach support has been disabled.
[I 120801 13:52:06 gateone:1800] No authentication method configured. All users will be ANONYMOUS
[I 120801 13:52:06 gateone:1876] Loaded plugins: bookmarks, help, logging, logging_plugin, notice, playback, ssh
[I 120801 13:52:06 gateone:2329] Listening on https://*:443/

At this point, gateone is running in the foreground and you can view as
connections occur and any errors. Pressing Ctrl If you conect to gateone
using your webbrowser, you are logged in as user ANONYMOUS and can
connect to any ssh host, either localhost or something remote.

If you edit /opt/gateone/server.conf, you can change authentication to
“pam” or “google”. Using pam will perform a Basic HTTP style
authenication requiring a system-level username and password. Using
google will log you in with your google account. Both of these “just
work” without complicated setup.

Running as a Non-Root

Before I put something like this in production, I wanted to apply some
additional security. First off, I want to see if I can get this to run
as a non-root user.

Since gateone ran as root user initially, it has files owned by rootOnly UID 0 can open ports below 1024.gateone may need permission to write to system directories

To solve the first one, I chowned the /opt/gateone directory to my
username. In the future, I’ll want to run it under its own user, but
I’ll use mine for now for simplicity. To solve the second and third, I
edited server.conf.

[email protected]:/opt/gateone$ sudo chown -R johnh:johnh .
[email protected]:/opt/gateone$ vi server.conf# change/add the following lines appropriatelyport = 2443session_dir = "/opt/gateone/tmp/gateone"pid_file = "/opt/gateone/tmp/gateone.pid"uid = 1000gid = 1000
[email protected]:/opt/gateone$ ./gateone.py
[W 120801 14:06:01 terminal:166] Could not import the Python Imaging Library (PIL) so images will not be displayed in the terminal
[W 120801 14:06:01 gateone:2232] dtach command not found.  dtach support has been disabled.
[I 120801 14:06:01 gateone:1802] No authentication method configured. All users will be ANONYMOUS
[I 120801 14:06:01 gateone:1876] Loaded plugins: bookmarks, help, logging, logging_plugin, notice, playback, ssh
[I 120801 14:06:01 gateone:2329] Listening on https://*:2443/

Authentication

Running as a lower uid, you can use authentication of None or “google”
without issue. If you use “pam”, you discover you can only login with
the username that gateone is running under. If you are the only intended
user of the service, this may not be an issue. But if you want to allow
other users, this becomes an issue. If you are fine with running as root
or using Google as your authentication provider, you can ignore this
next step.

Fortunately, pam is highly configurable. You aren’t required to
authenticate against shadow passwords. You can also authenticate against
db4 files with pam_userdb, msyql, or even htpasswd files. To start off,
I’m going to use htpasswd files. Note that Ubuntu doesn’t provide
pam_pwdfile.so by default. You need to install libpam-pwdfile (“sudo
apt-get install libpam-pwdfile”).

Note – in testing, I discovered gateone uses Crypt encryption while htpasswd defaults to MD5. Use -d to switch to crypt encryption.

[email protected]:/opt/gateone$ htpasswd -c -d users.passwd user1
New password:
Re-type new password:
Adding password for user user1
[email protected]:/opt/gateone$ cat users.passwd
user1:KKEPyZtUf9sadf9

Create a pam module called gateone under /etc/pam.d

[email protected]:/opt/gateone$ cat /etc/pam.d/gateone
#%PAM-1.0
# Login using a htpasswd file
@include common-sessionauth    
required pam_pwdfile.so          pwdfile /opt/gateone/users.passwdaccount 
required pam_permit.so

Modify server.conf to use pam and pam_service of gateone:

auth = "pam"
pam_service = "gateone"

Now start gateone and log in.

[email protected]:~/g1/gateone$ ./gateone.py
[W 120801 14:59:16 terminal:168] Could not import the Python Imaging Library (PIL) so images will not be displayed in the terminal
[W 120801 14:59:16 gateone:2577] dtach command not found.  dtach support has been disabled.
[I 120801 14:59:16 gateone:2598] Connections to this server will be allowed from the following origins: 'http://localhost https://localhost http://127.0.0.1 https://127.0.0.1 https://puppet2 https://127.0.1.1 https://puppet2:2443'
[I 120801 14:59:16 gateone:2023] Using pam authentication
[I 120801 14:59:16 gateone:2101] Loaded plugins: bookmarks, help, logging, logging_plugin, mobile, notice, playback, ssh
[I 120801 14:59:16 gateone:2706] Listening on https://*:2443/
[I 120801 14:59:16 gateone:2710] Process running with pid 32591
[I 120801 14:59:17 gateone:949] WebSocket opened ([email protected]).

One additional nice feature with authentication enabled is the ability
to resume sessions – even across different computers or browsers.

Reverse Proxy

(I failed on this part, but felt it was worth recording)

Once I got it working in single user mode, I wanted to go ahead and set
this up under a reverse proxy under Apache. This would allow me to
integrate it into my existing web server under a sub-directory.

First, I edited server.conf to use a URL prefix of /g1/

Second, I tried setting up a ReverseProxy in Apache.

# GateOne 
ProxySSLProxyEngine 
OnProxyPass /g1/ https://localhost:2443/g1/
ProxyPassReverse /g1/ https://localhost:2443/g1/
ProxyPassReverseCookieDomain localhost localhost
ProxyPassReverseCookiePath / /g1/

This almost worked. I had no errors, but the resulting page was
unreadable. However, at the bottom was a clue. “The WebSocket connection
was closed. Will attempt to reconnect every 5 seconds… NOTE: Some web
proxies do not work properly with WebSockets.” The problem was Apache
not properly proxying my websocket connection. People have managed to
get this working under nginx, but not Apache.

Searching for a solution led me to a similar question on ServerFault, an
apache-websocket module on github, and a websocket tcp proxy based on
that module.

  • http://serverfault.com/questions/290121/configuring-apache2-to-proxy-websocket
  • https://github.com/disconnect/apache-websocket
  • http://blog.alex.org.uk/2012/02/16/using-apache-websocket-to-proxy-tcp-connection/

In order to get this work, I’ll need to download and compile some code.
The apxs command requires the apache-prefork-dev package in
Debian/Ubuntu. Install it with “sudo apt-get install
apache-prefork-dev”.

Now we are ready to download the code and install the module:

[email protected]:~$ git clone https://github.com/disconnect/apache-websocket.git
Cloning into 'apache-websocket'..... done
[email protected]:~$ wget http://blog.alex.org.uk/wp-uploads/mod_websocket_tcp_proxy.tar.gz
[email protected]:~$ cd apache-websocket
[email protected]:~/apache-websocket$ sudo apxs2 -i -a -c mod_websocket.c*snip*
[email protected]:~/apache-websocket$ sudo apxs2 -i -a -c mod_websocket_draft76.c*snip*
[email protected]:~$ cd examples
[email protected]:~$ tar -xzvf ../../mod_websocket_tcp_proxy.tar.gzmod_websocket_tcp_proxy.c
[email protected]:~$ cd apache-websocket/examples/
[email protected]:~/apache-websocket/examples$ sudo apxs2 -c -i -a -I.. mod_websocket_tcp_proxy.c
*snip*
chmod 644 /usr/lib/apache2/modules/mod_websocket_tcp_proxy.so
[preparing module `websocket_tcp_proxy' in /etc/apache2/mods-available/websocket_tcp_proxy.load]
Enabling module websocket_tcp_proxy.To activate the new configuration, you need to run:service apache2 restart
[email protected]:~$

Before we restart, I want to remove my Proxy lines and replace them with
the mod_websocket_tcp_proxy lines.

SetHandler websocket-handler        
WebSocketHandler  /usr/lib/apache2/modules/mod_websocket_tcp_proxy.so tcp_proxy_init        
WebSocketTcpProxyBase64 on        
WebSocketTcpProxyHost 127.0.0.1        
WebSocketTcpProxyPort 2443        
WebSocketTcpProxyProtocol base64

Despite all this, I was still unable to get this to work. I even
attempted using the web root (/) as my location. If the Location matches
and your HTTP request is handled by mod_websocket, you get a 404. If
you use Proxy, then your websocket request is handled by mod_proxy.
Mod_proxy wins out over Location matches. Perhaps you can modify
gateone code to have one URL for the web interface and one for
websockets (or maybe it’s already in place and we just need to know),
but I don’t see a way at this time to get this working under Apache. I
may be able to work with the gateone author and the
mod_websocket_tcp_proxy.c author to come up with a solution. Or I
could try installing nginx. In the meantime, I can continue to run Open
Gate as a non-root user on a non-standard port. Alternatively, I could
find a wrapper to bring port 443 to 2443.

Puppet Dashboard and selinux

Tags: puppet selinux

Once I got a rough handle on setting up Puppet, I decided to get Puppet
Dashboard
working on my puppetmaster (a Centos 6 server) to have a
sort of web interface to view puppet statuses. Since I already have the
puppetlabs repositories setup from when I installed puppet, this was
almost as simple as running “yum install puppet-dashboard”.

You then go to /usr/share/puppet-dashboard and follow the install
guide
to get the database working. Then you can run it locally using
the “sudo -u puppet-dashboard ./script/server -e production” command or
starting the service (service puppet-dashboard start). I recommed
running it manually the first few times to make sure everything is
working.

At this point, I was able to browse to http://puppetmaster:3000/ and
view a nice looking dashboard with no hosts checked in. I then made sure
to add the following to puppetmaster’s puppet.conf [master] section and
restart puppet.

reports = http, store
reporturl = http://localhost:3000/reports/upload

I performed a “puppet agent –test” on one of my puppets, and here is
where I ran into trouble. Everything appeared to work, but no report or
“pending task” showed up in the Dashboard. I ran the Dashboard locally
so I could see the http request coming in. No request. I double checked
my configuration, everything looked good.

Finally, I ran my puppetmaster in debug/no-daemonize mode.
(/usr/sbin/puppetmaster –debug –no-daemonize) so I could watch what
was happening. However, in this mode, it worked fine. I ran
/usr/sbin/puppetmaster without debugging and it still worked. The
reports would get submitted to dashboard if I ran puppetmaster directly,
but not if I started it with the init scripts.

I couldn’t find any differences between how the init script was starting
puppetmaster vs me starting it manually. However, I did come across this
entry in /var/log/messages:

 Jul 25 10:15:16 puppetmasterj puppet-master[11988]: Compiled catalog for puppet2.lab in environment production in 1.16 seconds
 Jul 25 10:15:17 puppetmasterj puppet-master[11988]: Report processor failed: Permission denied - connect(2)

Which led me to the following entry in audit.log

 type=AVC msg=audit(1343225819.078:1582): avc:  denied  { name_connect } for  pid=11988 comm="puppetmasterd" dest=3000 scontext=unconfined_u:system_r:puppetmaster_t:s0 tcontext=system_u:object_r:ntop_port_t:s0 tclass=tcp_socket

This was an selinux issue. I quickly ascertained that turning enforcing
off in selinux allowed my reports to get through. I couldn’t find anyone
else online encountering this issue. However, many people disable
selinux enforcing early on and I guess the cross-section of
puppet-dashboard users and those using selinux enforcing is somewhat
small.

How to fix this? There is a set of python programs called “audit2why”
and “audit2allow” as part of the policycoreutils-python package. These
tools will parse entries from the audit.log and either explain why an
action was denied or provide a solution. You can get these tools by
doing a “yum install policycoreutils-python”.

Now we can use audit2allow to parse our audit.log error. You’ll want to
run the tool, paste in your log entry, and then hit Ctrl+D on a blank
line.

 [[email protected] tmp]# audit2allow -m puppetmaster
 type=AVC msg=audit(1343232143.497:1617): avc:  denied  { name_connect } for  pid=12552 comm="puppetmasterd" dest=3000 scontext=unconfined_u:system_r:puppetmaster_t:s0 tcontext=system_u:object_r:ntop_port_t:s0 tclass=tcp_socket
 module puppetmaster 1.0;
 require {
     type puppetmaster_t;
     type ntop_port_t;
     class tcp_socket name_connect;
 }
 #============= puppetmaster_t ==============
 allow puppetmaster_t ntop_port_t:tcp_socket name_connect;

The above gives you a textual view of a module you can create to allow
puppetmaster to make an outbound connection. audit2allow will even
generate that module with the -M option.

 [[email protected] tmp]# audit2allow -M puppetmaster
 type=AVC msg=audit(1343232143.497:1617): avc:  denied  { name_connect } for  pid=12552 comm="puppetmasterd" dest=3000 scontext=unconfined_u:system_r:puppetmaster_t:s0 tcontext=system_u:object_r:ntop_port_t:s0 tclass=tcp_socket
 ******************** IMPORTANT ***********************
 To make this policy package active, execute:
 semodule -i puppetmaster.pp

That generated the module “puppetmaster.pp”. Despite the pp extension,
this is not a puppet manifest, but an selinux binary module. Let’s
install it.

 [[email protected] tmp]# semodule -i puppetmaster.pp

With that, puppetmaster can communicate with dashboard and reports are
flowing. The only remaining thing left to do is to file a bug report. As
it happened, someone had posted a bug report similar to this for
documentation on the puppetdb project. I decided to append to that
issue, but I suggested migrating the issue to the main puppet project.
Issue #15567.

Using puppet to install djbdns

This is a basic walkthrough of getting a slightly complex “step by step
to install” program like djbdns to install under puppet (in this case,
under Ubuntu 12.04). It shows building the manifest, testing it, and
some possible gotchas.

I am generally following the guide put together by Higher Logic[1], with
a few changes of my own.

Step 1: Installation
I use the dbndns fork of djbdns, which has a few patches installed that
djbdns lacks. In fact, the djbdns package in Debian/Ubuntu is a virtual
package that really install dbndns. To install it normally, you would
type “sudo apt-get install dbndns”. This would also install daemontools
and daemontools-run. However, we’ll also need make and ucspi-tcp.

We’re going to do this the puppet way. I’m assuming my puppet
configuration in in /etc/puppet, node manifests are in
/etc/puppet/nodes, and modules are in /etc/puppet/modules.

a. Create the dbndns module with a package definition to install

sudo mkdir -p /etc/puppet/modules/dbndns/manifests
    sudo vi /etc/puppet/modules/dbndns/manifests/init.pp

        class dbndns {
            package {
                    dbndns:
                    ensure => present;





ucspi-tcp:
                    ensure => present;

make:
                    ensure => present;
            }

}

b. Create a file for your node (ie: puppet2.example.net)

sudo vi /etc/puppet/nodes/puppet2.example.net.pp

        node    ‘puppet2.lab.example.net’ {
            include dbndns
        }





c. Test
Ok, to test on your puppet client, run “sudo puppet agent –test”

[email protected]:~# sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340213237’
    notice: /Stage[main]/Dbndns/Package[dbndns]/ensure: created
    notice: Finished catalog run in 3.39 seconds







Here we can see our dbndns package installed. But is it running? Well,
djbdns uses daemontools, which runs svscan, and some searching online
shows that in Ubuntu 12.04/Precise, this is now an upstart job. svscan
is not running. So let’s make it run. Add the following to your init.pp
(within the module definition):

define the service to restart

        service { “svscan”:
                ensure  => “running”,
                provider => “upstart”,
                require => Package[“dbndns”],
        }




Now back on puppet2, let’s test it.

[email protected]:~# sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340213237’
    notice: /Stage[main]/Dbndns/Service[svscan]/ensure: ensure changed
‘stopped’ to ‘running’
    notice: Finished catalog run in 0.47 seconds







We now told puppet to ensure that svscan is running. The “provider”
option tells it to use upstart instead of /etc/init.d/ scripts or the
service command. Also, we make sure that it doesn’t attempt to start
svscan unless dbndns is already installed.

Now we have daemontools running, but we haven’t got it start our tinydns
service yet. To do that, we need to create some users and configure the
service.

Step 2: Create users

Going back to our guide, our next step is to create users. We can do
that in puppet as well.
    # Users for the chroot jail
    adduser –no-create-home –disabled-login –shell /bin/false dnslog
    adduser –no-create-home –disabled-login –shell /bin/false
tinydns
    adduser –no-create-home –disabled-login –shell /bin/false
dnscache



So in our init.pp module file, we need to define our users:

user { “dnslog”:
            shell => “/bin/false”,
            managehome => “no”,
            ensure => “present”,
        }

    user { “tinydns”:
            shell => “/bin/false”,
            managehome => “no”,
            ensure => “present”,
        }

    user { “dnscache”:
            shell => “/bin/false”,
            managehome => “no”,
            ensure => “present”,
        }















Back on puppet2, we can give that a test.

[email protected]:~$ sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340215757’
    notice: /Stage[main]/Dbndns/User[dnscache]/ensure: created
    notice: /Stage[main]/Dbndns/User[tinydns]/ensure: created
    notice: /Stage[main]/Dbndns/User[dnslog]/ensure: created
    notice: Finished catalog run in 0.86 seconds
    [email protected]:~$ cat /etc/passwd | grep dns
    dnscache:x:1001:1001::/home/dnscache:/bin/false
    tinydns:x:1002:1002::/home/tinydns:/bin/false
    dnslog:x:1003:1003::/home/dnslog:/bin/false













So far, so good. Now we have to do the configuration, which will require
executing some commands.

Step 3 – Configuration
Our next step are the following commands:

Config

    tinydns-conf tinydns dnslog /etc/tinydns/ 1.2.3.4
    dnscache-conf dnscache dnslog /etc/dnscache 127.0.0.1
    cd /etc/dnscache; touch /etc/dnscache/root/ip/127.0.0
    mkdir /etc/service ; cd /etc/service ; ln -sf /etc/tinydns/ ; ln -sf
/etc/dnscache

The first two commands create our service directories. Authoratative
tinydns is set to listen on 1.2.3.4 and dnscache is set to listen on
127.0.0.1. The 3rd command creates a file that restricts dnscache to
only respond to requests from IPs starting with 127.0.0. This is isn’t
necessary, but the challenge is interesting.





What we want to do first is see if /etc/tinydns and /etc/dnscache exist
and if not, run the -conf program. We also need to know the IP address.
Fortunately, puppet provides this as a variable “$ipaddress”. Try
running the “facter” command.

Puppet has a property call creates that is ideal. If the directory
specified by creates does not exist, it will perform the associated
commands. Here are our new lines:

exec { “configure-tinydns”:
            command => “/usr/bin/tinydns-conf tinydns dnslog
/etc/tinydns $ipaddress”,
            creates => “/etc/tinydns”,
            require => Package[‘dbndns’],
    }



exec { “configure-dnscache”:
            command => “/usr/bin/dnscache-conf dnscache dnslog
/etc/dnscache 127.0.0.1”,
            creates => “/etc/dnscache”,
            require => Package[‘dbndns’],
    }



Thos will configure tinydns and dnscache, and then we can restrict
dnscache

file { “/etc/dnscache/root/ip/127.0.0”:
            ensure => “present”,
            owner => “dnscache”,
            require => Exec[“configure-dnscache”],
    }



Then, we need to create the /etc/service directory and bring tinydns and
dnscache under svscan’s control.

    file { “/etc/service”:
            ensure => “directory”,
            require => Package[“dbndns”],
    }




file { “/etc/service/tinydns”:
            ensure => “link”,
            target => “/etc/tinydns”,
            require => [ File[‘/etc/service’],
Exec[“configure-tinydns”], ],
    }



file { “/etc/service/dnscache”:
            ensure => “link”,
            target => “/etc/dnscache”,
            require => [  File[‘/etc/service’],
Exec[“configure-dnscache”]  ],
    }



And our tests:

[email protected]:~$ sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340218775’
    notice: /Stage[main]/Dbndns/Exec[configure-dnscache]/returns:
executed successfull
    notice:
/Stage[main]/Dbndns/File[/etc/dnscache/root/ip/127.0.0]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/service/dnscache]/ensure:
created
    notice: /Stage[main]/Dbndns/Exec[configure-tinydns]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/tinydns]/ensure:
created
    notice: Finished catalog run in 0.59 seconds
    [email protected]:~$ ls /etc/service/tinydns/root/
    add-alias  add-alias6  add-childns  add-host  add-host6  add-mx 
add-ns  data  Makefile
    [email protected]:~$ ps ax | grep supervise
     7932 ?        S      0:00 supervise dnscache
     7933 ?        S      0:00 supervise log
     7934 ?        S      0:00 supervise tinydns
     7935 ?        S      0:00 supervise log


















Doing a dig www.example.net @localhost returns 192.0.43.10, so dnscache
works.

Now, let’s check tinydns. No domains are configured yet, so let’s put
example.com in there. Edit /etc/tinydns/root/data and put these lines in
it, substituting 10.100.0.178 for your own “public” IP address.

&example.com::ns0.example.com.:3600 
Zexample.com:ns0.example.com.:hostmaster.example.com.:1188079131:16384:2048:1048576:2560:2560
 +ns0.example.com:10.100.0.178:3600

Then “make” the data.cdb file:

cd /etc/tinydns/root ; sudo make

Now test:

[email protected]:/etc/tinydns/root$ dig ns0.example.com @10.100.0.178

; <<>> DiG 9.8.1-P1 <<>> ns0.example.com @10.100.0.178
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25433
    ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL:
0
    ;; WARNING: recursion requested but not available




;; QUESTION SECTION:
    ;ns0.example.com.               IN      A

;; ANSWER SECTION:
    ns0.example.com.        3600    IN      A       10.100.0.178

;; AUTHORITY SECTION:
    example.com.            3600    IN      NS      ns0.example.com.

Ok, for a final test, let’s remove everything and run it again.

sudo service svscan stop
    sudo apt-get purge daemontools daemontools-run ucspi-tcp dbndns
    sudo rm -rf /etc/service /etc/tinydns /etc/dnscache
    sudo userdel tinydns
    sudo userdel dnslog
    sudo userdel dnscache

Let’s do this:






[email protected]:~$ sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340220032’
    notice: /Stage[main]/Dbndns/Service[svscan]/ensure: ensure changed
‘stopped’ to ‘running’
    err: /Stage[main]/Dbndns/Exec[configure-dnscache]/returns: change
from notrun to 0 failed: /usr/bin/dnscache-conf dnscache dnslog
/etc/dnscache 127.0.0.1 returned 111 instead of one of [0] at
/etc/puppet/modules/dbndns/manifests/init.pp:47
    notice: /Stage[main]/Dbndns/User[dnscache]/ensure: created
    notice: /Stage[main]/Dbndns/User[tinydns]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/service]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/service/dnscache]: Dependency
Exec[configure-dnscache] has failures: true
    warning: /Stage[main]/Dbndns/File[/etc/service/dnscache]: Skipping
because of failed dependencies
    notice: /Stage[main]/Dbndns/User[dnslog]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/dnscache/root/ip/127.0.0]:
Dependency Exec[configure-dnscache] has failures: true
    warning: /Stage[main]/Dbndns/File[/etc/dnscache/root/ip/127.0.0]:
Skipping because of failed dependencies
    notice: /Stage[main]/Dbndns/Exec[configure-tinydns]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/tinydns]/ensure:
created
    notice: Finished catalog run in 0.98 seconds


















Looks like we had something fail. Oops! configure-dnscache failed. We
see that the user dnscache and tinydns were created after. So we need to
make sure that the users are created before we can configure the
service. This needs to happen to tinydns as well as dnscache. Good thing
we did this test so it doesn’t bite us in the future. Let’s adjust our
init.pp

exec { “configure-tinydns”:
                command => “/usr/bin/tinydns-conf tinydns dnslog
/etc/tinydns $ipaddress”,
                creates => “/etc/tinydns”,
                require => [ Package[‘dbndns’], User[‘dnscache’],
User[‘dnslog’] ],
        }



exec { “configure-dnscache”:
                command => “/usr/bin/dnscache-conf dnscache dnslog
/etc/dnscache 127.0.0.1”,
                creates => “/etc/dnscache”,
                require => [ Package[‘dbndns’],  User[‘dnscache’],
User[‘dnslog’] ],
        }



Also, let’s go ahead and run our commands above to get rid of everything
again.

[email protected]:~$ sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340220641’
    notice: /Stage[main]/Dbndns/Service[svscan]/ensure: ensure changed
‘stopped’ to ‘running’
    notice: /Stage[main]/Dbndns/User[dnscache]/ensure: created
    notice: /Stage[main]/Dbndns/User[tinydns]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/service]/ensure: created
    notice: /Stage[main]/Dbndns/User[dnslog]/ensure: created
    notice: /Stage[main]/Dbndns/Exec[configure-dnscache]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/dnscache]/ensure:
created
    notice:
/Stage[main]/Dbndns/File[/etc/dnscache/root/ip/127.0.0]/ensure: created
    notice: /Stage[main]/Dbndns/Exec[configure-tinydns]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/tinydns]/ensure:
created
    notice: Finished catalog run in 1.05 seconds
















Everything looks good, but when we run “ps ax | grep svscan” we don’t
see svscan running. So we check /var/log/syslog and see this

Jun 20 19:31:35 puppet2 kernel: [ 9646.348251] init: svscan main
process ended, respawning
    Jun 20 19:31:35 puppet2 kernel: [ 9646.359074] init: svscan
respawning too fast, stopped

If we start it by hand, it works, so what happened? /etc/service didn’t
exist yet.


[email protected]:~$ sudo service svscan start
    svscan start/running, process 9726
    [email protected]:~$ ps ax | grep supervise
     9730 ?        S      0:00 supervise dnscache
     9731 ?        S      0:00 supervise log
     9732 ?        S      0:00 supervise tinydns
     9733 ?        S      0:00 supervise log





Let’s fix that.

define the service to restart

        service { “svscan”:
                ensure  => “running”,
                provider => “upstart”,
                require => [ Package[“dbndns”], File[“/etc/service”] ]
        }




Now, let’s give it a go:

[email protected]:~$ sudo puppet agent –test
    info: Retrieving plugin
    info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
    info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
    info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
    info: Caching catalog for puppet2.lab.example.net
    info: Applying configuration version ‘1340220885’
    notice: /Stage[main]/Dbndns/User[dnscache]/ensure: created
    notice: /Stage[main]/Dbndns/User[tinydns]/ensure: created
    notice: /Stage[main]/Dbndns/File[/etc/service]/ensure: created
    notice: /Stage[main]/Dbndns/Service[svscan]/ensure: ensure changed
‘stopped’ to ‘running’
    notice: /Stage[main]/Dbndns/User[dnslog]/ensure: created
    notice: /Stage[main]/Dbndns/Exec[configure-dnscache]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/dnscache]/ensure:
created
    notice:
/Stage[main]/Dbndns/File[/etc/dnscache/root/ip/127.0.0]/ensure: created
    notice: /Stage[main]/Dbndns/Exec[configure-tinydns]/returns:
executed successfully
    notice: /Stage[main]/Dbndns/File[/etc/service/tinydns]/ensure:
created
    notice: Finished catalog run in 1.24 seconds
    [email protected]:~$ ps ax | grep svscan
    10613 ?        Ss     0:00 /bin/sh /usr/bin/svscanboot
    10615 ?        S      0:00 svscan /etc/service
    10639 pts/0    S+     0:00 grep –color=auto svscan
    [email protected]:~$ ps ax | grep supervise
    10630 ?        S      0:00 supervise dnscache
    10631 ?        S      0:00 supervise log
    10632 ?        S      0:00 supervise tinydns
    10633 ?        S      0:00 supervise log
    10641 pts/0    S+     0:00 grep –color=auto supervise


























Excellent! We now have a working puppet class that will install puppet,
configure it, and get it up and running. At this point, we don’t have
any records being served by tinydns, but it wouldn’t be hard to push a
file to /etc/tinydns/root/data and execute a command to perform the
make. In my case, I will be using VegaDNS’s update-data.sh[2] to pull
the data remotely.

Here is our completed modules/dbndns/init.pp:


class dbndns {

package {
                dbndns:
                ensure => present;

ucspi-tcp:
                ensure => present;

make:
                ensure => present;
        }

define the service to restart

        service { “svscan”:
                ensure  => “running”,
                provider => “upstart”,
                require => [ Package[“dbndns”], File[“/etc/service”] ]
        }




user { “dnslog”:
                        shell => “/bin/false”,
                        managehome => false,
                        ensure => “present”,
                }



user { “tinydns”:
                        shell => “/bin/false”,
                        managehome => false,
                        ensure => “present”,
                }



user { “dnscache”:
                        shell => “/bin/false”,
                        managehome => false,
                        ensure => “present”,
                }



exec { “configure-tinydns”:
                command => “/usr/bin/tinydns-conf tinydns dnslog
/etc/tinydns $ipaddress”,
                creates => “/etc/tinydns”,
                require => [ Package[‘dbndns’], User[‘dnscache’],
User[‘dnslog’] ],
        }



exec { “configure-dnscache”:
                command => “/usr/bin/dnscache-conf dnscache dnslog
/etc/dnscache 127.0.0.1”,
                creates => “/etc/dnscache”,
                require => [ Package[‘dbndns’],  User[‘dnscache’],
User[‘dnslog’] ],
        }



file { “/etc/dnscache/root/ip/127.0.0”:
                ensure => “present”,
                owner => “dnscache”,
                require => Exec[“configure-dnscache”],
        }



file { “/etc/service”:
                ensure => “directory”,
                require => Package[“dbndns”],
        }


file { “/etc/service/tinydns”:
                ensure => “link”,
                target => “/etc/tinydns”,
                require => [ File[‘/etc/service’],
                                        Exec[“configure-tinydns”],
                                ],
        }





file { “/etc/service/dnscache”:
                ensure => “link”,
                target => “/etc/dnscache”,
                require => [  File[‘/etc/service’],
                                        Exec[“configure-dnscache”]
                                ],
        }





}


[1]
http://higherlogic.com.au/2011/djbdns-on-ubuntu-10-04-server-migration-from-bind-and-zone-transfers-to-secondaries-bind/
[2] https://github.com/shupp/VegaDNS/blob/master/update-data.sh

Sending messages using xmpppy

In continuing working with XMPP and Python, I managed to get xmpppy
working.

This proved to be particularly tricky because xmpppy uses some outdated
modules like md5 and sha.  It also relies on some dns functions which no
longer seem to work.

I was able to use the sample script “xsend.py” that the project
provides.  The big thing is that I had to specify talk.google.com and
the port number directly in the connect command.

If I did not specify the servername, I would get the error:
 An error occurred while looking up _xmpp-client._tcp.example.com

NOTE: DNS is configured correctly for the domain I was practicing with,
but the xmpppy code was not able to resolve it.

To run the script, simply execute;

./xblast.py [email protected]  text to send

The Script:

!/usr/bin/python

$Id: xsend.py,v 1.8 2006/10/06 12:30:42 normanr Exp $

import sys,os,xmpp,time

if len(sys.argv) < 2:
    print “Syntax: xsend JID text”
    sys.exit(0)

tojid=sys.argv[1]
text=’ ‘.join(sys.argv[2:])

jidparams={}

jidparams[‘jid’]=’[email protected]
jidparams[‘password’] = ‘123456’

jid=xmpp.protocol.JID(jidparams[‘jid’])
cl=xmpp.Client(jid.getDomain(),debug=True)

con=cl.connect((‘talk.google.com’,5222),  use_srv=False)

if not con:
    print ‘could not connect!’
    sys.exit()
print ‘connected with’,con
auth=cl.auth(jid.getNode(),jidparams[‘password’],resource=jid.getResource())
if not auth:
    print ‘could not authenticate!’
    sys.exit()
print ‘authenticated using’,auth

cl.SendInitPresence(requestRoster=0)   # you may need to uncomment

this for old server
id=cl.send(xmpp.protocol.Message(tojid,text))
print ‘sent message with id’,id

time.sleep(1)   # some older servers will not send the message if you
disconnect immediately after sending

cl.disconnect()


Nagios and XMPP

I found that someone has written a perl script geared towards sending
alerts from Nagios to XMPP usernames.

http://www.gridpp.ac.uk/wiki/Nagios_jabber_notification

I have downloaded this, but have not yet got it working as of yet, but
it does look promising.  It took me a while to update all the
dependencies for it, but if those were in place, the installation itself
is rather simple.  That is, the notification script works, but I haven’t
actually configured nagios as of yet.

This script has a shortcoming in that it only accepts the username
portion of the JID and not the domain name — and this means that
notifications can only be sent between users of the same domain.

To illustrate, [email protected] can not send a message to
[email protected], but user1 and user2 on example.net can send to each
other.

# this command will fail:
[email protected]:/usr/local/bin$ ./notify_by_jabber.pl
[email protected] testing 
# while this one works
[email protected]:/usr/local/bin$ ./notify_by_jabber.pl yourtech
testing


And in the perl script, you have to specify the login credentials and
server you’re connecting to:

## Configuration
# my $username = ‘[email protected]’;   # does not work
my $username = ‘system’;
my $password = “password”;
my $resource = “nagios”;
## End of configuration




my $len = scalar @ARGV;
if ($len ne 2) {
   die “Usage…n $0 [jabberid] [message]n”;
}
my @field=split(/,/,$ARGV[0]);
#————————————




# Google Talk & Jabber parameters :

my $hostname = ‘talk.google.com’;
my $port = 5222;
# componentname is the second half of your JID:
my $componentname = ‘example.com’;
my $connectiontype = ‘tcpip’;
my $tls = 1;




Jabbering about Python

I am getting more and more into the idea of using XMPP for sending
messages.  XMPP (also known as jabber) is a very open protocol.  Anyone
can throw up a server, and by publishing a few DNS records, your server
can interact with any other XMPP server out there (that is open).  For a
popular example, anyone can throw up their server with their domain and
send messages to someone using Google Chat.

I have a customer that has their domain email hosted by Google apps for
domains, which also means that each person has Google Talk, which is
based off of XMPP.  For a monitoring scenario, I decided I’d rather have
alerts go to an installed Google Talk client instead of to their email. 
This would make alerts more noticeable, but less intrusive to their
Inbox.  Google would also tie all their messages together (chat and
email) for searching later. So my idea was to have monitoring server
send the email alert to a local user, and have it piped through a
program that would send the XMPP alert.  In the future, perhaps an XMPP
bot could also accept commands to control the monitoring system.

I wanted to create a program called “jabblast.py” which would accept
either standard input, or a command line argument, and send a message to
a pre-defined list of recipients.



echo “sent from stdin” | jabblast.py
  jabblast.py -m “sent from a command line argument”

Looking at the python and XMPP options out there, I came across three
that looked promising. 

# xmpppy
  # sleekxmpp
  # twisted (the “words” extension)

I was really expecting to find exactly what I wanted to do already
written.  However, it seems anyone writing an XMPP script is creating a
bot that runs continuously and would need modified for a one-time
event.

Sleekxmpp was actually the easiest to understand and work with, however
the project could be in danger of abandonment.   Twisted looks the most
advanced, and I am drawn to it because it seems to offer the most
versatility.  In my mind, I suspect that if I learn twisted for this, I
can just keep using it for all the other stuff I want to do.  However,
it might be “too much” for this simple of an operation.  xmpppy is the
middle ground.  The project seems relatively active and popular. 

The problem with all three projects is a severe lack of documentation —
for twisted, this lack is primarily on the xmpp side.  One thing very
well hidden is how to specify the server name you want to connect to. 
Ideally, this is provided through a DNS record lookup, but that method
eliminates experimentation with local network ip addresses.   Sleek
never even approaches the concept that you might want your XMPP program
to end.   It has a call for disconnecting, but when I call that, sleek
immediately tries to reconnect.

I’m going to go ahead and create my program using all three methods and
decide later which I like best.

For these examples, I have hardcoded the login credentials, as well as
the “to” into the source code itself.  In the real world, I should be
pulling that from a configuration file, either XML or YAML (and
potentially from a remote web page).  Secondly, I only list one “to”.  I
believe that going from sending a single message to sending multiple
messages should be a matter of a for loop.  Some libraries might even
offer a feature to send to multiple recipients at once.

For my first example, I have created a script in sleekxmpp.  I wrote
this by looking at the EchoBot example found here.  The script will
send a message and then hang for a while after disconnecting before it
finally exits.  I don’t know why.  If you enable logging, an error is
thrown after you pass in the disconnect command.  But otherwise, a
relatively simple bit of code, easy to expand upon.

#!/usr/bin/env python

import sys
import logging
import sleekxmpp

# Uncomment the following line to turn on debugging
#logging.basicConfig(level=logging.DEBUG, format=”%(levelname)-8s
%(message)s”)

def main() :
  # configure jabber id credentials and to
  to = ‘[email protected]
  myjid = ‘[email protected]/blastbot’
  mysecret = ‘password’



args = sys.argv[1:]
  if not args:
    message = sys.stdin.read()
  else:
    message = sys.argv[1]



# print to, message
  # sys.exit()

bot = BlastBot(myjid, mysecret, to, message)
  bot.run()

class BlastBot :

def __init__(self, jid, password, to, message) :
    self.message = message
    self.to = to
    self.xmpp = sleekxmpp.ClientXMPP(jid, password)
    self.xmpp.add_event_handler(“session_start”,
self.handleXMPPConnected)
    self.xmpp.add_event_handler(“disconnected”,
self.handleXMPPDisconnected)




def run(self) :
    self.xmpp.connect()
    self.xmpp.process(threaded=False)

def handleXMPPDisconnected(self, event) :
    sys.exit()

def handleXMPPConnected(self, event) :
    self.xmpp.sendMessage(self.to, self.message)
    self.xmpp.disconnect(self)

if __name__ == “__main__” :
  main()

Expanding a raw image file

On one of my customer’s devices, the backup software requires that the
backups go to a separate partition (or drive).  However, the customer
only has one raid array and the bulk of the space is in /home.   To work
around this limitation, I created a raw image file called backup.img,
which gets mounted as /backup.   After the software performs its local
backup, I use duplicity to backup /backup remotely to a backup server at
my location (with encryption).

Today I got an alert that /backup was running low on space.  It was an
80GB image and 61GB was in use, leaving only 15GB free.  Now, this
amount of free space should last quite a while.  However, the software
(cPanel)  has a known issue for years that the 80% limit is hardcoded
into the program.  I can change this, but every time cPanel updates, it
overwrites that change.

So to be proactive, I decided to go ahead and increase the image size.

In order to increase the size of an image, you simply unmount your raw
image and use the dd command.

# Increase by ~20GB
dd if=/dev/zero bs=1M count=20480 >> backup.img
# 20,480 is 20,480 MB or ~20GB

# check the filesystem
/sbin/e2fsck -f backup.img
# resize the filesystem
/sbin/resize2fs backup.img
# check the filesystem again
e2fsck -f backup.img               




Learning Python

I am reasonably familiar with Perl and PHP languages.  Recently though,
I have taken it upon myself to learn Python.  I’ve played with Python a
few times, and it seems to be very powerful with lots of useful
functionality under the hood.

For some basic practice, I recommend Learn Python The Hard Way,
which is free book by a reputable Python author, Zed Shaw.  He makes the
book itself available for free as a PDF download or you can purchase it
directly.  The book walks you through a number of examples, each one
building upon what you learned in the previous exercise.  You are
required to enter each exercise in by hand.  This uses the principle
that if you learn better by writing things down, you’ll learn better by
typing it in.  Copying and pasting completely bypasses that manual entry
learning process, so if you want to learn, I suggest you follow Shaw’s
instructions.  I found the book rather insightful, but I often felt that
certain aspects were not being properly explained to me.  To be fair,
I’m only up to exercise 15 in a 52 exercise book.

However, I also came across another way to learn Python.  Google has had
instructor Nick Parlante teach Python in a classroom setting.  The
class was video taped and put online along with the associated
learning aids (primarily a group of python files).  The class was taught
in 7 segments over 2 days.  So if you go to the class website, you can download the learning aids, and then follow along with
the lecture videos. 

I like this method better than following the static book.  Since it was
taught in an interactive classroom, it goes at a pace where you can keep
up with it from your own computer.  Also, students in the class were
able to ask questions, which allows the information to be represented in
a different way.  Finally, at the end of each segment, there are some
sample files you can open up in your editor.  Each file has a skeleton
template.  In comments, it tells you what a segment is supposed to do,
and you have to implement it in code based on what you learned while
watching the video.  The sample file also includes some bare bones “unit
tests” that call your functions, passing it input data and comparing it
to an expected output.  These unit tests will show what they received
from your function and what they expected.  If the two matched, you have
succeeded. 

I like these practice samples mainly because it immediately forces you
to apply the knowledge you learned.  You do have to think about it and
do some back and forth troubleshooting on your own code.  Since they
provided the unit tests, you can get instant feedback on if you’re doing
it correctly or not.

If you want to learn Python and can dedicate an hour on any given day to
it, I recommend the Google course.  If you want to go in smaller 10-20
minute blocks, try using Learn Python The Hard Way.  Of course, you can
pause the videos and come back to them, but I think it’s much better to
go through the entire video, following along on your own system as you
go.

We have moved!

Starting around Memorial Day, we have been busy working on moving our
office to our new location in Bedford, PA.  We have now completed that
move and wanted to let everyone know our new address.

Our new address for correspondence is:

YourTech, LLC
300 Cumberland Rd
Bedford, PA 15522 

Securing Apache2

A client wanted to make files available available to the web browser
from within their LAN and a handful of static IPs without requiring any
sort of username or password.  This is the web equivalent of a shared,
read only folder.  This is no big issue, you can create an .htaccess
file like so:

Order deny,allow
Deny from all
Allow from 10.10.0.1/16
Allow from 127.0.0.1/32
Allow from 1.1.1.1/32

However, they would also like to access it from a remote location with a
username and password.

First, we need to create a password file.  In the old days, you would
use “AuthType Basic”, but a more secure method is the Digest method.  To
use this, you must have the auth_digest module loaded into your Apache
configuration.  If you are running a Debian or Ubuntu version, you can
do this by executing “sudo a2enmod auth_digest“.  Using digest
authentication prevents your username and password from being sent in
the clear (however, I always recommend that any site requiring
authentication should utilize https).

Next,  you create your htdigest file:

htdigest -c .htdigest authname username

When prompted, you would enter a password for username.

Finally, you need to modify your .htaccess file to allow either method:

Order deny,allow
Deny from all
AuthName “authname”
AuthType Digest
AuthUserFile /var/www/.htpasswd
require valid-userAllow from 10.10.0.1/16
Allow from 127.0.0.1/32
Allow from 1.1.1.1/32
Satisfy Any

Now, a user can come from the 10.10.x.x network, localhost, or 1.1.1.1
without requiring authentication.  If they later come from an
unrecognized IP, they can enter their username and password and be
granted access.