Tips & Tricks for Technologists & System Administrators | About & Contact

Category: System Administration


Centos pip python install error

Posted 18th September in Linux, Shell. No Comments

While attempting to install Thumbor on a CentOS server I recently had the following error message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
------------------------------------------------------------
/usr/bin/pip run on Thu Sep 18 21:07:45 2014
Getting page https://pypi.python.org/simple/pycrypto/
URLs to search for versions for pycrypto in /usr/lib64/python2.6/site-packages:
* https://pypi.python.org/simple/pycrypto/
Analyzing links from page https://pypi.python.org/simple/pycrypto/
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.0.1.tar.gz#md5=4d5674f3898a573691ffb335e8d749cd (from https://pypi.python.org/simple/pycrypto/), version: 2.0.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.1.0.tar.gz#md5=1d3eb04f06e6f09a080bc37fb019f9bf (from https://pypi.python.org/simple/pycrypto/), version: 2.1.0
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.2.tar.gz#md5=4f0ed728b14b98f09120cb2ec461ec98 (from https://pypi.python.org/simple/pycrypto/), version: 2.2
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.3.tar.gz#md5=2b811cfbfc342d83ee614097effb8101 (from https://pypi.python.org/simple/pycrypto/), version: 2.3
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.1.tar.gz#md5=c2a1404a848797fb0806f3e11c29ef15 (from https://pypi.python.org/simple/pycrypto/), version: 2.4.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.tar.gz#md5=274fa44c30a320d56460a93fdd95e702 (from https://pypi.python.org/simple/pycrypto/), version: 2.4
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz#md5=783e45d4a1a309e03ab378b00f97b291 (from https://pypi.python.org/simple/pycrypto/), version: 2.5
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda (from https://pypi.python.org/simple/pycrypto/), version: 2.6.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.tar.gz#md5=88dad0a270d1fe83a39e0467a66a22bb (from https://pypi.python.org/simple/pycrypto/), version: 2.6
Using version 2.6.1 (newest of versions: 2.6.1, 2.6, 2.5, 2.4.1, 2.4, 2.3, 2.2, 2.1.0, 2.0.1, 2.0.1)
Downloading/unpacking pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda

  Running setup.py egg_info for package pycrypto

    running egg_info
    writing pip-egg-info/pycrypto.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pycrypto.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pycrypto.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/pycrypto has version 2.6.1, which satisfies requirement pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda
Installing collected packages: pycrypto

  Found existing installation: pycrypto 2.0.1

    Uninstalling pycrypto:

      Removing file or directory /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
      Successfully uninstalled pycrypto

  Running setup.py install for pycrypto

    Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed
    running install
    running build
    running build_py
    running build_ext
    running build_configure
    checking for gcc... gcc

    checking whether the C compiler works... yes

    checking for C compiler default output file name... a.out

    checking for suffix of executables...

    checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

    configure: error: cannot run C compiled programs.

    If you meant to cross compile, use `--host'.

    See `config.log' for more details

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

        core.setup(**kw)

      File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

        dist.run_commands()

      File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

        self.run_command(cmd)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

        return _install.run(self)

      File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

        self.run_command('build')

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

        raise RuntimeError("autoconf error")

    RuntimeError: autoconf error

    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_py

running build_ext

running build_configure

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `--host'.

See `config.log' for more details

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

    core.setup(**kw)

  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

    self.run_command(cmd)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

    return _install.run(self)

  File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

    self.run_command('build')

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

    raise RuntimeError("autoconf error")

RuntimeError: autoconf error

----------------------------------------

  Rolling back uninstall of pycrypto

  Replacing /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/usr/lib/python2.6/site-packages/pip/commands/install.py", line 271, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 1185, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 592, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

It essentially boils down to:

1
2
3
4
checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

Weird, I have gcc and all compile programs installed.

It took me a fair time of troubleshooting, but I finally figured out it was because it was attempting to build this in /tmp, which I have set to mount at noexec for security purposes. This disallows execution of programs in this directory.

Running

1
mount -oremount,exec /tmp

Allowed it to run without issue.


SSH – Unspecified GSS failure

Posted 10th September in CentOS, Linux, Shell, System Administration. No Comments

Recently came across a problem with one system authentication to another via ssh.

I added the public ssh key to ~/.ssh/authorized_keys entry. Changed ownership to the proper user and also chmod 600 ~/.ssh/authorized_keys. Still no dice.

Using ssh -vvvv the following error returned on pubkey authentication:

1
2
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

Well, that is pretty non-specific.

Since permission problems are the number one issue I have with getting ssh authentication working, and how I thought I set permissions right, my mind then wondered if SELinux permissions were causing a problem. Most providers disable SELinux right off the bat because of the ‘problems’ it causes, but some don’t. It turned out, this one has SELINUX=enforcing. So, let’s fix the SELinux permissions:

1
/sbin/restorecon -r /root/.ssh

This sets the context as follows:

1
2
# ls -Z authorized_keys
-rw-------. root root unconfined_u:object_r:ssh_home_t:s0 authorized_keys

SSH should now authenticate.


How to Turn PHP into an RPM The Easy Way with FPM

So you have a custom PHP binary that you wish to distribute across multiple servers.

The “old” way to do this is to create a bash script in order to pull down the source code, run ./configure and then make; make install. But this is 2013, we have a much more civilized way to create this package now, so that you can distribute that RPM across multiple nodes or even include it in your own yum repository.

We’re going to use FPM (Effing Package Manager) to do the job for this. It is a little bit confusing because PHP also includes the acronym FPM for FastCGI Process Manager. Our package manager FPM is a ruby gem, so install the prerequisites first (CentOS 6 directions here), along with fpm:

1
2
# yum install ruby ruby-devel rubygems
# gem install fpm

To start, compile PHP the way you normally would, ie:

1
2
3
4
5
6
'./configure'  '--program-prefix=' '--prefix=/usr' \
'--exec-prefix=/usr' '--bindir=/usr/bin' '--enable-ftp' \
'--enable-calendar' '--with-libxml-dir=/usr' '--enable-mbstring' \
'--enable-mbregex' '--with-gd' '--enable-zip' '--with-mcrypt' \
'--with-mhash' '--with-libdir=lib64' '--enable-fpm' '--with-fpm-user=www' \
'--with-fpm-group=www' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd'

Then make it as normal.

1
2
# make
... Lots of output

Now instead of running make install like normal, we tell it to output to another directory, just some temporary directory on the server:

(Note, if compiling another program, the switch to make it install into another directory may be different, such as DESTDIR=/tmp/installdir)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
make install INSTALL_ROOT=/tmp/installdir
Installing PHP SAPI module:       fpm
Installing PHP CLI binary:        /tmp/installdir/usr/bin/
Installing PHP CLI man page:      /tmp/installdir/usr/share/man/man1/
Installing PHP FPM binary:        /tmp/installdir/usr/sbin/
Installing PHP FPM config:        /tmp/installdir/etc/
Installing PHP FPM man page:      /tmp/installdir/usr/share/man/man8/
Installing PHP FPM status page:      /tmp/installdir/usr/share/fpm/
Installing build environment:     /tmp/installdir/usr/lib/build/
Installing header files:          /tmp/installdir/usr/include/php/
Installing helper programs:       /tmp/installdir/usr/bin/
  program: phpize
  program: php-config
Installing man pages:             /tmp/installdir/usr/share/man/man1/
  page: phpize.1
  page: php-config.1
/usr/local/src/php-5.3.25/build/shtool install -c ext/phar/phar.phar /tmp/installdir/usr/bin
ln -s -f /usr/bin/phar.phar /tmp/installdir/usr/bin/phar
Installing PDO headers:          /tmp/installdir/usr/include/php/ext/pdo/

Now PHP is happily installed in /tmp/installdir. Which is great, but we can’t use it until it we package it up for regular installation. Run the FPM command to pack it all up:

1
2
3
4
5
# fpm -s dir -t rpm -n php-fpm -v 5.3.25 -C /tmp/installdir \
   -p php-fpm_VERSION_ARCH.rpm -d openssl -d pcre -d bzip2 -d curl \
   -d libjpeg -d libpng -d freetype -d gmp -d libmcrypt -d libmhash \
   -d libxml2 usr/bin usr/share/man/man1 usr/sbin etc \
   usr/share/man/man8 usr/share/fpm usr/lib/build usr/include/php

What do the options mean?

I’ll go through these options:

  • -s dir: source is a directory
  • -t rpm: we are creating an rpm
  • -n php-fpm: we are creating a package with the name php-fpm
  • -v 5.3.25: the version of the rpm (so you can manage upgrades)
  • -C /tmp/installdir: the base location to take the files from
  • -p php-fpm_VERSION_ARCH.rpm: fpm will replace the VERSION and ARCH macros automatically with your version number and architecture
  • -d openssl -s pcre …: these are all of the rpm dependencies that you’ll want on the machines installing the package. If you don’t include the proper ones (to match your source system, and which are included in ./configure, you’ll get library errors when you try to launch the program.
  • usr/bin usr/share/man/man1 usr/sbin etc usr/share/man/man8 usr/share/fpm usr/lib/build usr/include/php: the directories you wish to include in your package file

When all is done, you should have your rpm file located in /tmp/installdir. You can install it directly on any system with rpm -i <rpm name>

Before you package up your files, you can include any other files you want. For example, you could include the init script or php.ini file. But if you are managing the system using puppet or chef, it would be better to manage this configuration file through there rather than from the rpm file. This works really well as a ‘building block’ for your programs when you add these custom rpms into your own repository. That way, you can have your system install custom versions of the binaries where applicable but still have the niceties that come from an systems automation with puppet or chef.

Enjoy!


Simple Guide To Signing RPMs with FPM

I’ve been using the excellent fpm (Effing package manager!) tool for automatically generating rpms from source (for example, creating a custom compiled php-fpm binary and then wrapping it for install in an rpm for distribution from our own repository). FPM also creates .debs and other binaries as you need them.

To ensure we have a secure infrastructure we wanted to make sure that the rpms are correctly signed so that the gpg check can be completed on them when installed.

First, generate the gpg public/private key pair:

1
gpg --gen-key

Fill out the required information. The entropy generation step may take a long time to complete.

Export a text version of the public key with this command, for later use in your repository:

1
gpg --export -a 'Dave Drager' > ~/RPM-GPG-KEY-reponame

fpm docs state that you just need to add the –rpm-sign to your fpm command. However, this generated the following error for me:

1
2
error: You must set "%_gpg_name" in your macro file {:level=>:error}
Pass phrase check failed {:level=>:error}

The reason for this is that it uses the settings located in your ~/.rpmmacros file. Find your key ID:

1
2
3
4
5
6
# gpg --list-keys
/user/.gnupg/pubring.gpg
------------------------
pub   2048R/94E8C1F6 2013-04-24
uid                  Dave Drager <ddrager@xxx>
sub   2048R/94E8C1F6 2013-04-24

In this sample, the KEY_ID would be “94E8C1F6″

1
2
3
4
%_signature gpg
%_gpg_path /path/to/.gnupg
%_gpg_name KEY_ID
%_gpgbin /usr/bin/gpg

From here, the fpm command runs without issue.

1
2
3
4
# fpm -s dir -t rpm -n php-fpm-xda -v 5.2.23 -C /tmp/installdir -p php-fpm-xda_VERSION_ARCH.rpm -d openssl -d pcre -d bzip2 -d curl -d libjpeg -d libpng -d freetype -d gmp -d libmcrypt -d libmhash -d libxml2 --rpm-sign usr/bin usr/share/man/man1 usr/sbin etc usr/share/man/man8 usr/share/fpm usr/lib/build usr/include/php
Enter pass phrase:
Pass phrase is good. {:level=>:error}
Created rpm {:path=>"php-fpm-xda_5.2.23_x86_64.rpm"}

This is an alias of the rpm --addsign command.

Now let’s check the signature on this rpm.

1
2
# rpm --checksig php-fpm-xda_5.2.23_x86_64.rpm
php-fpm-xda_5.2.23_x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING KEYS: (MD5) PGP#94E8C1F6)

Whoops, it doesn’t seem to be signed properly. Well, that is because the rpm system doesn’t know to trust this GPG key yet. Import with the following command:

1
rpm --import ~/RPM-GPG-KEY-reponame

Note that this is the plain text file from before. Your RPM system knows to trust this key, and now if you verify the file:

1
2
# rpm --checksig php-fpm-xda_5.2.23_x86_64.rpm
php-fpm-xda_5.2.23_x86_64.rpm: rsa sha1 (md5) pgp md5 OK

This RPM is now ready for use in a repo. In the .repo file, use the following to ensure it follows the correct signature:

1
2
gpgcheck=1
gpgkey=http://repo.xxxx.com/RPM-GPG-KEY-reponame

All key names have been changed to protect the innocent.


Track TCP and UDP Connections With Zabbix

There is a very easy way to track opened TCP connections in Zabbix. Although it does not come built in to the Zabbix agent, it is simple enough to get. This method will allow you track the open number of TCP and UDP connections, as well as track TCP connections in TIME_WAIT.

TCP UDP Sockets Zabbix

Rather than using

1
grep

to count numbers in netstat, which can take an extraordinary amount of time on busy servers, I’ve used /proc/net/sockstat to get this data.

Add the following parameters to zabbix_agentd.conf:

1
2
3
4
5
6
7
8
UserParameter=sockstat.sockets,cat /proc/net/sockstat|grep sockets|cut -d' ' -f 3
UserParameter=sockstat.tcp.inuse,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 3
UserParameter=sockstat.tcp.orphan,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 5
UserParameter=sockstat.tcp.timewait,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 7
UserParameter=sockstat.tcp.allocated,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 9
UserParameter=sockstat.tcp.mem,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 11
UserParameter=sockstat.udp.inuse,cat /proc/net/sockstat|grep UDP:|cut -d' ' -f 3
UserParameter=sockstat.udp.mem,cat /proc/net/sockstat|grep UDP:|cut -d' ' -f 5

Then you can either manually import these UserParameters into Zabbix, or use this template (if you’re using Zabbix 2.x).

Download the sockstat Zabbix template

It includes the items and a graph for the socket stats. Enjoy!


Using rsync on an Alternate SSH Port or with OpenSSH Keys

Posted 9th October in Linux, Shell, System Administration. 4 Comments

Best practices state that you should run ssh on an non-standard port. Unfortunately some programs use port 22 by default and it isn’t obvious what the switch is to change this port.

One of these programs is the eminently useful rsync. It states how to do this in the man file, but it is hidden and non-obvious. You simply add the option via the -e command to pass ssh options. -e is the shell rsync uses to connect to a remote host.

Using rsync on an alternate ssh port

1
rsync -avz -e "ssh -p $port" username@ip:/path/to/files/ /local/files/

Using rsync with an openssh key

1
rsync -avz -e "ssh -i /path/to/private/key" username@ip:/path/to/files/ /local/files/

Or with both an alternate port and openssh key:

1
rsync -avz -e "ssh -i /path/to/private/key -p $port" username@ip:/path/to/files/ /local/files/

This can be used in a bash script if you set the $port variable or directly on the command line by using the set port instead of $port.

Here are some other useful pages regarding using rsync. If you don’t use rsync for remote file transfer – I highly recommend it. The transfer speed is much faster than scp over ssh.


Simple Sysadmin Trick: Using tcpdump To Sniff Web Server Traffic

Posted 2nd October in Linux, Shell, System Administration. 1 Comment

Sometimes, you just have to look into the raw data to see what your web server is doing. The logs might not show you enough detail or you suspect something is going on which is just not shown in the log files. Or, as in my case, logging is turned off because of too much activity.

The excellent tcpdump utiliy comes to the rescue here. I recommend you get more familiar with the tcpdump man page. Here is the command you can use, in a nutshell:

1
tcpdump -nl -w - -i eth1 -c 500 port 80|strings

or alternatively with just tcpdump (Thanks Chris!):

1
tcpdump -nl -s 0 -A -i eth1 -c 500 port 80

Your command line will print out all traffic exiting your server from port 80, headers and all. Lets look at the options in more detail.

  • -n: Don’t convert addresses (i.e., host addresses, port numbers, etc.) to names.
  • -l: Make stdout line buffered. Useful if you want to see the data while capturing it.
  • -w: Write the raw packets to file rather than parsing and printing them out. (Sent to stdout)
  • -i: Interface you want to sniff on, usually eth0 or eth1, but depends on your system.
  • -c: Number of packets to capture
  • port: port 80, duh :)
  • -A: Print each packet (minus its link level header) in ASCII.
  • -s: size

Now, depending on your web server configuration, you will probably have gzipped content which comes out as garbled characters. To strip all that out, just pipe it through strings.

The output will look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET /about/comment-page-1 HTTP/1.0
Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, */*
User-Agent: Mozilla/2.0 (compatible; MSIE 3.02; Windows CE; 240x320)
Referer: http://quittingsoda.com/about/comment-page-1#comment-6637
Host: quittingsoda.com
Cookie: comment_author_xxx=sakyjartory; comment_author_email_26e707905b5fd6e7139333eb1dab208f=olfaexxxx; comment_author_url_26e707905b5fd6e7139333eb1dab208xxx
HTTP/1.1 200 OK
Date: Wed, 03 Oct 2012 01:49:19 GMT
Server: Apache/2
X-Powered-By: PHP/5.2.17
X-Pingback: http://quittingsoda.com/xmlrpc.php
Vary: Accept-Encoding,User-Agent
Connection: close
Content-Type: text/html; charset=UTF-8
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head profile="http://gmpg.org/xfn/11">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>About This Site | Quitting Soda</title>

Let me know if you have any suggestions for improving this simple command line program for sniffing your web server traffic! Netcat is an essential tool that every linux administrator should get to know better.