400% DNS Speedup

When I placed my server at QuadraNet, they gave me two IP addresses for resolving DNS. I dutifully placed both of them in my resolv.conf file, and thought nothing more of it.

That is until they had problems resolving a name or two. I remembered long ago trying namebench on a home computer, and I wondered how it would fare on my server.

The answer is: it worked amazingly well. Namebench found me two new DNS servers, the fastest of which is 408.9% faster than the ones QuadraNet provided me with. I’m guessing that both of the faster public DNS servers it found for me are physically in the same building as QuadraNet, but I could be wrong. Anyway, the exciting thing is just how much faster they are!

It’s important to remember that resolv.conf allows you to specify up to three nameservers. By default it goes through the list, from first to last, to resolve a hostname. If a server can’t tell you the name or times out, your system will automatically move on to the next nameserver.

So you really should have three nameservers in your resolv.conf list.

Posted in Linux, Uncategorized | Leave a comment

Upgrading to Ubuntu 11.04 on Rackspace

Currently, in addition to running my own hardware, I use both the Amazon and Rackspace clouds. Rackspace unfortunately hasn’t added new images in a while. Therefore, in order to run Ubuntu 11.04 (Natty) I had to start with 10.10 (Maverick).

Ubuntu’s upgrade instructions are pretty clear. In fact, of the three steps given only two are necessary:

  • sudo apt-get install update-manager-core
  • sudo do-release-upgrade

As long as you start with Ubuntu 10.10, Prompt is already set to normal, so there is no need to edit any files.

These two commands are easy enough to execute, but you have to confirm many steps. That’s okay if you only have to do the upgrade once. But I’m constantly bringing up and taking down servers, so the whole process needed to be automated.

Installing the update-manager-core can be easily automated:

apt-get install --assume-yes update-manager-core

It’s the do-release-upgrade part that is harder to automate. There are two types of prompts presented when you do the upgrade. The first ones come from the upgrade program itself, the others come from the dpkg/apt subsystem.

You can quiet all of the dpkg messages by calling do-release-upgrade like this:

DEBIAN_FRONTEND=noninteractive /usr/bin/do-release-upgrade

But that unfortunately won’t stop all the messages that you need to confirm.

My next thought was to combine that with the program “yes”, a small unix tool which outputs the string “y” over and over.

So we’d have something like:

yes | DEBIAN_FRONTEND=noninteractive /usr/bin/do-release-upgrade

But unfortunately that doesn’t work, instead the program complains: “Must be connected to a terminal.”

So finally I had to put together this little expect program in Python.

#!/usr/bin/python

import pexpect
import sys

child = pexpect.spawn(
    '''/bin/bash -c "DEBIAN_FRONTEND=noninteractive /usr/bin/do-release-upgrade"''',
    timeout = 10 * 60  # 10 minutes
    )
child.logfile = sys.stdout

child.expect(r'''Continue running under SSH?.*Continue \[yN\]''')
child.sendline('y')

child.expect(r'''If you run a firewall.*To continue please press \[ENTER\]''')
child.sendline('')

child.expect(r'''Fetching and installing the upgrade.*Continue \[yN\]  Details \[d\]''')
child.sendline('y')

child.expect(r'''Remove obsolete packages?.*Continue \[yN\]  Details \[d\]''')
child.sendline('y')

# We prefer to do our own rebooting
child.expect(r'''If you select 'y' the system will be restarted.*Continue \[yN\]''')
child.sendline('N')

Save the above file with the name “upgrade_release.py”. You can then run it directly. Or, if you use fabric to manage the creation of all of your new servers, you can add it to your fabfile like so:

def upgrade_dist():
    output = run("lsb_release -r")
    version = output.partition(":")[2].strip()
    if version == "10.10":
        print("Going to upgrade Ubuntu...")
        sudo("aptitude install --assume-yes update-manager-core python-pexpect")

        put("fabfile/upgrade_release.py", "/tmp")
        sudo("/usr/bin/python /tmp/upgrade_release.py")

        print("Done upgrading Ubuntu - Need to sleep while rebooting")
        reboot(60)
        # Line below necessary because of ssh caching bug in fabric
        fabric.state.connections = fabric.state.HostConnectionCache()
    else:
        print("Not going to upgrade Ubuntu")
Posted in Cloud, Python | Tagged , , , , | Leave a comment

Rackspace Cloud Trailing

Rackspace Cloud seems to be getting more and more behind these days.  Most of the Linux distributions they offer are out of date.

Debian 6.0 (Squeeze) was released February 6, 2011. Rackspace only offers Debian 5.

CentOS is at 5.6, Rackspace offers only 5.4 and 5.5.

RedHat Enterprise Linux 6.1 is out – Rackspace offers  only 5.4 and 5.5.

Fedora is at version 15 – but Rackspace offers only Fedora 14.

Oracle Linux is at version 6 – Rackspace offers R5U4 and R5U3.

Ubuntu is on 11.04 – but the latest Rackspace offers is 10.10.

Gentoo and Arch are both current – but they can’t be more than 3% of the Linux market combined.

I had an online chat with their support department. From what I can piece together from that chat and this Rackspace blog entry, Rackspace is apparently waiting until after they switch to Citrix XenServer to start offering new releases.

You can, of course, install any of these earlier releases and then upgrade them.  But that takes time and is a bit of a pain. Current customers will probably be willing to do this.

But I imagine there are lots of new customers every day trying out the “cloud” for the first time.  And those who are using the current release of their favorite distribution – upon seeing that Rackspace doesn’t offer it – will probably turn to Amazon and never look back.

Which is a shame, because Rackspace has a lot of great features: in particular they offer great support and it’s incredibly easy to get started with them.

But they need to offer the latest versions of all the major Linux releases.

In my next post I’ll show you how you can use Python to automate updating Ubuntu 10.10 (Maverick) to 11.04 (Natty).

Posted in Cloud | Tagged , | Leave a comment