When doing a PostgreSQL database server upgrade, you first need to backup all of the databases because the format changes from version to version.

Postgresql documentation states you need to run the command:

pg_dumpall > outputfile

I ran this, and received the error message:

pg_dumpall: could not connect to database "template1": FATAL: role "root" does not exist

So after some head scratching and googling without any results; I noticed that I had created a user for postgres; namely ‘postgres’. I sued to that user and ran the command - what do you know - it works fine! I’m glad that error message was so clear.

So, the thing that the documentation does not state - make sure you are running under the Postgresql user before running the backup command.

I recently formatted my laptop and installed Windows first, using half of the hard drive, and then installing Ubuntu 7.10 on the other half. It had been a while since I tried Ubuntu - it has a come a long way - but that is another story.

The install worked fine, however at the end, it just sort of hung while installing grub. GREAT. I reboot and it kicks me to a (grub) standard prompt.

It took me a while to figure this out, but you can manually boot Ubuntu via the grub prompt. The tricky part is finding out the right commands since your system is totally inaccessible.

You can find your available hard drive name by typing:

> root ( <tab>

pressing tab will list your available hard drives and partitions and hopefully your Ubuntu ext3 partition.

Continue setting the root boot partition. This includes your partition with all /boot files. For example, mine was installed onto the root / filesystem, and not a separate filesystem.

> root (hd0,2)
>

This sets your root that grub uses.

You then need to set the kernel. Use:

> kernel /boot/vmlin <tab>

Tab will show you the available files to use. Also you can use this at any level to explore your filesystem. So for example:

> kernel /boot/vmlinuz-2.6.15-20-386

But wait before you hit enter!

You will get a pivot root error - the kernel doesn’t know where the rest of your file system is. In a file called device.map in your /boot directory, this location should be specified. In my case, it is listed as /dev/sda3. This is important for your kernel. Fortunately, GRUB has a ‘cat’ command you use to get the text output of this file. We also set this to a read-only filesystem - Ubuntu takes care of setting it back to rw when it boots.

So the final kernel line is:

> kernel /boot/vmlinuz-2.6.15-20-386 root=/dev/sda3 ro
>

Now we need to set the initrd file - hopefully you by now know to use to find this if you do not know it off of the top of your head:

> initrd /initrd.img-2.6.15-20-386
>

And finally, type ‘boot’ to begin the boot process:

> boot

Hopefully ubuntu will boot for you now without problems. Again, if you get a pivot root or ‘unable to mount root VFS’ error - you need to check the root=/dev/sda3 line part of the grub commands. This means that the kernel could not find your main filesystem.

Once you are logged into Ubuntu - use the following command to regenerate menu.lst for you. Once this is rebuilt, you will be presented with the normal Grub menu and you won’t need to follow the process above!

sudo update-grub

If you need to compile ffmpeg for PHP (for example, to allow PHP to process video similar to how YouTube does) - then follow this Howto from Nazly. For a CentOS machine with custom PHP, it works great.

Ffmpeg PHP Extension Compile Instructions

The technology has been around for a while to do unattended OS installs, using kickstart on redhat for example. These installs are typically done remotely. Kickstart is basically done off of an image file, which can be inserted via a floppy, CD, or now a USB Keychain drive.

Another technology that has been around a while is pxeboot. Using pxeboot, you can boot images directly from a supported network card. Most network cards have a built in bootp interface. The network card first looks for a bootp (aka dhcp) server to assign an IP address and load an image. It then uses the trivial ftp protocol to look for a boot image, called pxelinux. Then pxelinux loads the correct installer image. So the process looks like this:
Power on->Boot from Network->Bootp->Tftp->pxelinux->Boot Image Loads
After loading the boot image, the system is on its own as far as doing the install. Best of all, this means that as long as the OS has an unattended boot image, it can be loaded from this method. For anyone running a datacenter, this is excellent! It provides a way to do these installs from the next city, the next state, or even halfway across the world!

Setting up the bootp server

The bootp protocol is supported in Redhat’s (and I imagine others) dhcp server.

For this exercise, I am going to assume you are using yum. But, you can do these installs the same with sudo apt-get install or installing the rpms/debs manually.

First, install dhcpd:
yum install dhcpd
Then setup the dhcp server
# vi /etc/dhcpd.conf

group {
next-server 10.0.2.2; # name of your TFTP server
filename "pxelinux.0"; # name of the bootloader program
}

You can see from this code that we are setting up the install network on the 10.0.2.x private network.

The image filename the dhcp server tells the network card to boot off is pxelinux.0.

TFTP Server

The TFTP server is necessary to grab the initial image files from. I recommend either tftp-hpa for linux, or tftpd32 for Windows. For the rest of this article, I will assume that you are using tftp-hpa on linux.

After compiling tftp-hpa from source, enable it adding/editing /etc/xinet.d/tftp like so:

service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -s /tftpboot
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}

Restart xinetd. The tftp server is now serving files from /tftpboot/

PXELinux

PXELinux is a software package from H. Peter Anvin. It loads other OS images from its initial boot interface. See his site for more information on this powerful package. Download the pxelinux package and extract pxelinux.0 to /tftpboot/. Then create the directory /tftpboot/pxelinux.cfg.

When the system boots, it looks for an configuration file (similar to syslinux syntax) in the pxelinux.cfg directory. First it tries to match by the MAC address of the network card, the it goes by a hexadecimal representation of the IP address, then it loads default. for example, if the Ethernet MAC address is 88:99:AA:BB:CC:DD and the IP address 192.0.2.91, it will try (in order):

/tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd
/tftpboot/pxelinux.cfg/C000025B
/tftpboot/pxelinux.cfg/C000025
/tftpboot/pxelinux.cfg/C00002
/tftpboot/pxelinux.cfg/C0000
/tftpboot/pxelinux.cfg/C000
/tftpboot/pxelinux.cfg/C00
/tftpboot/pxelinux.cfg/C0
/tftpboot/pxelinux.cfg/C
/tftpboot/pxelinux.cfg/default

Notice the mac address has 01- preceeding it, and each colon is replaced by a - dash.

Now, to make sure I don’t inadvertantly install over a good os, I make sure to keep a default file in this directory with the following contents:

default linux
label linux
localboot 0

This tells the system to boot off of the hard drive.

A CentOS Remote install

CentOS is one of the easier OSes to install via this method. I will use this for an example during this article.

We first need a CentOS image to boot from and use for the install image. Lets put this image in the tftpboot directory in a location called images/CentOS/4.4_i386. Rsync is a good choice to use, but you can really use anything you want to transfer the image (off of a CD for example). Here is a set of commands that works for me:

mkdir -p /tftpboot/images/CentOS/4.4_i386
cd /tftpboot/images/CentOS/
rsync -r rsync://mirror.linux.duke.edu::centos/centos/4.4/os/i386 4.4_i386

Then copy the boot images from the CentOS directory to the tftpboot directory:

cd /tftpboot/
cp /tftpboot/images/CentOS/4.4_i386/i386/isolinux/initrd.img initrd_centos_4.4.img
cp /tftpboot/images/CentOS/4.4_i386/i386/isolinux/vmlinuz vmlinuz_centos_4.4

Its ok to leave those guys alone. If you would ever need to update them to include other drivers, you can do so but that is beyond the scope of this article.

Lets create a /tftpboot/Kickstart/ks.cfg file now:

lang en_US
langsupport en_US
keyboard us
mouse none --device null

%post
rpm --import /usr/share/doc/centos-release-4/RPM-GPG-KEY-centos4
echo "co:2345:respawn:/sbin/agetty -h -t 60 ttyS0 9600 vt102" >> /etc/inittab
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config

For a comprehensive guide on the options for this file, check out the Redhat 9 Kickstart options HOWTO which is still good for our purposes.

In the above code we are loading the OS from a nfs share.

If nfs is not installed you’ll need to do that first:

yum install nfs
chkconfig nfs on

Add the following line to /etc/exports

/tftpboot 10.0.2.0/255.255.255.0(sync)

Start up nfs

service nfs start

Putting it all together

Now, lets use the above to put this automated installer to work!

You’ll need to set up your hardware on the network along with the installation system. You’ll need to set up the boot order to let the machine boot off the network firstly; then HD, CD or floppy next. If it boots to the disk first, the machine will never contact the automated installer and will just try (and fail, if the HD is blank) to boot off the the HD.

You will also need to mark down the MAC Address of the network card the server will be booting off of. This will be retrieved from the actual hardware; off of the boot menu; or as a last resort you can boot it off of your DHCP server and then check the log for when this server hits the installer machine.

Lets create our pxelinux boot file. It should be named based off of the Ethernet MAC Address. For example: /tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd

Note that the mac address has an 01 in the very beginning of it; sometimes this is not included when you write down the mac address of the system.

default linux
serial 0,9600n8
label linux
kernel vmlinuz_centos_4.4
append ksdevice=eth0 console=tty0 console=ttyS0,9600n8 load_ramdisk=1 initrd=initrd_centos_4.4.img network ks=nfs:10.0.2.2:/tftpboot/Kickstart/ks.cfg

You may have noticed that this file specifies a few things. The initrd and vmlinuz files we copied, handle the kickstart of the image. /tftpboot/Kickstart/ks.cfg is the kickstart configuration file and is needed to do the actual unattended install. If you do not need remote serial console access during this install, you should remove the string “console=ttyS0,9600n8″ which simply adds output to the serial port.

That is it! Now that all of the files are set up; you can boot/power cycle the server. This is what should happen:

1. Server boots and looks for DHCP server on its network interface. Installer server responds and assigns server an IP address
2. Server begins looking for a pxeboot configuration for what to do. It (hopefully) finds the file /tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd and boots using this
3. It loads vmlinuz_centos_4.4 and the initrd initrd_centos_4.4.img.
4. Kickstart config is told to load from the nfs share nfs:10.0.2.2:/tftpboot/Kickstart/ks.cfg
5. Kickstart install begins like a normal Kickstart install from a floppy or CD install

Here is a catch: After the installation starts; remove the pxeboot.cfg file. If you do not remove/move this file, when the machine reboots after the install it will just do the remote installation again, resulting in an infinite install loop.

Bonus: Windows Unattended Installation

In order to perform this same process but with Windows; we use the Unattended installation system.

Follow the directions for basic installation of this system.

Once you have it installed; copy :unattended install:/linuxboot/tftpboot/* to /tftpboot/

Then create a pxeboot.cfg file with the following lines:

default unattended
label unattended
kernel bzImage
# Add options (z_user=..., z_path=..., etc.) to this line.
append initrd=initrd_unattended

The Unattended installer will then work off its own installer system; which will automatically provision Windows versions - I’ve tested 2003 but it should also work for 2000 and XP installs.

In order to customize the installs further, you can use nLite to slipstream service packs and hotfixes; as well as make other changes to the default install.

Conclusion

Using this method allows you to deploy lots of installs with minimal datacenter presence. Once you have the hardware set up and ready to go; you just need to create the pxeboot config file and power cycle the machine. This has great benefits if you are far away from the datacenter; or what to deploy a lot of machines without doing manual installs. If you have any other solutions that will automate installations in a similar manner; please let me know!

If you are a system administrator, you should dread any time you use the normal “ls” command and receive a strange error in return. That is a sure sign that your machine has been hacked and ls has been replaced by an unknown program. Gnist blog has a nicely written step-by-step list of what you can do to track back an intruder.

In his case it sounds like the original owner of the machine may have used a weak root password on his machine, allowing for someone to crack it and break into the box. People, don’t forget to use a hard-to-brute force password, and finally disable root login s. Instead, use normal users and sudo.

Finally, if you are doing this in a legal setting and preparing for a possible future court case, don’t forget to make an image of the drive (using dd or Norton Ghost) before your analysis; otherwise the intruder can get your evidence thrown out.

Some other security resources:

• APF + BFD
• OSSEC
• Sleuthkid
• chkrootkit