Tips & Tricks for Technologists & System Administrators | About & Contact

Category: Other Code


Track TCP and UDP Connections With Zabbix

There is a very easy way to track opened TCP connections in Zabbix. Although it does not come built in to the Zabbix agent, it is simple enough to get. This method will allow you track the open number of TCP and UDP connections, as well as track TCP connections in TIME_WAIT.

TCP UDP Sockets Zabbix

Rather than using

1
grep

to count numbers in netstat, which can take an extraordinary amount of time on busy servers, I’ve used /proc/net/sockstat to get this data.

Add the following parameters to zabbix_agentd.conf:

1
2
3
4
5
6
7
8
UserParameter=sockstat.sockets,cat /proc/net/sockstat|grep sockets|cut -d' ' -f 3
UserParameter=sockstat.tcp.inuse,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 3
UserParameter=sockstat.tcp.orphan,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 5
UserParameter=sockstat.tcp.timewait,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 7
UserParameter=sockstat.tcp.allocated,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 9
UserParameter=sockstat.tcp.mem,cat /proc/net/sockstat|grep TCP|cut -d' ' -f 11
UserParameter=sockstat.udp.inuse,cat /proc/net/sockstat|grep UDP:|cut -d' ' -f 3
UserParameter=sockstat.udp.mem,cat /proc/net/sockstat|grep UDP:|cut -d' ' -f 5

Then you can either manually import these UserParameters into Zabbix, or use this template (if you’re using Zabbix 2.x).

Download the sockstat Zabbix template

It includes the items and a graph for the socket stats. Enjoy!


The Easy CIDR Cheatsheet

Posted 9th February in Configurations, Other Code, System Administration. Comments Off

Even though I’ve been working with Classless Inter-Domain Routing (henceforth known as CIDR) for years now, I always need a bit up a help remember how many addresses are in each block and how many sub-blocks fit into larger blocks. I have the following printed out for easy reference, and here it is for your geeky enjoyment:

CIDR        Total number    Network             Description:
Notation:   of addresses:   Mask:
--------------------------------------------------------------
/0          4,294,967,296   0.0.0.0             Every Address
/1          2,147,483,648   128.0.0.0           128 /8 nets
/2          1,073,741,824   192.0.0.0           64 /8 nets
/3          536,870,912     224.0.0.0           32 /8 nets
/4          268,435,456     240.0.0.0           16 /8 nets
/5          134,217,728     248.0.0.0           8 /8 nets
/6          67,108,864      252.0.0.0           4 /8 nets
/7          33,554,432      254.0.0.0           2 /8 nets
/8          16,777,214      255.0.0.0           1 /8 net (Class A)
--------------------------------------------------------------
/9          8,388,608       255.128.0.0         128 /16 nets
/10         4,194,304       255.192.0.0         64 /16 nets
/11         2,097,152       255.224.0.0         32 /16 nets
/12         1,048,576       255.240.0.0         16 /16 nets
/13         524,288         255.248.0.0         8 /16 nets
/14         262,144         255.252.0.0         4 /16 nets
/15         131.072         255.254.0.0         2 /16 nets
/16         65,536          255.255.0.0         1 /16 (Class B)
--------------------------------------------------------------
/17         32,768          255.255.128.0       128 /24 nets
/19         16,384          255.255.192.0       64 /24 nets
/19         8,192           255.255.224.0       32 /24 nets
/20         4,096           255.255.240.0       16 /24 nets
/21         2,048           255.255.248.0       8 /24 nets
/22         1,024           255.255.252.0       4 /24 nets
/23         512             255.255.254.0       2 /24 nets
/24         256             255.255.255.0       1 /24 (Class C)
--------------------------------------------------------------
/25         128             255.255.255.128     Half of a /24
/26         64              255.255.255.192     Fourth of a /24
/27         32              255.255.255.224     Eighth of a /24
/28         16              255.255.255.240     1/16th of a /24
/29         8               255.255.255.248     5 Usable addresses
/30         4               255.255.255.252     1 Usable address
/31         2               255.255.255.254     Unusable
/32         1               255.255.255.255     Single host
--------------------------------------------------------------
Reserved Space:
	0.0.0.0/8	
	127.0.0.0/8
	192.0.2.0/24
	10.0.0.0/8
	172.16.0.0/12
	192.168.0.0/16
	169.254.0.0/16

Of course I’m not the first one to come up with this. Modified based on the cheat sheet from Samat Jain.

Let me know if you have any improvements or suggestions.


Experimenting with Pascal on Ubuntu

Posted 21st February in Code Samples, Other Code, Ubuntu. Comments Off

I’ve been busy lately on a number of projects, one of which is a programming class I am currently taking. The class itself is interesting, we are learning about the different types of programming languages. For our latest project, we were tasked with writing a simple program in Pascal. Pascal isn’t used too much any more since it lacks some of the features that most modern languages have, but it is good to know at least a little bit about it in case you ever run across some old Pascal programs in the wild.

The syntax for pascal is a bit verbose, that is the main complaint about it. There are a number of others, but that is beyond the scope of this howto.

Installing The Pascal Compiler on Ubuntu

Installing Pascal in modern Ubuntu is a cinch. The Free Pascal Compiler, or fpc, is all that you need to get started. It works great on 32-bit or 64-bit systems. Install with:

1
sudo apt-get install fpc

Any prerequisites will automatically download and install along with fpc.

Getting Started in Pascal

To test the compiler let’s start with a simple Hello World program. Open up hello.pas and enter:

1
2
3
4
5
6
7
program Hello;

begin

Writeln('Hello World');

end.

Compile with fpc hello.pas and run:

1
2
dave@cerberus:~/Pascal$ ./hello
Hello World

Selection Sort in Pascal

Now that we’ve verified it is running, I’m going to show you the code that I wrote for my program. Basically we were asked to Selection Sort two arrays of varying length. Apparently one of the (bad) features of Pascal originally was that you needed to declare the length of the array which made it a pain to work with them.

In this situation it is just two arrays so it isn’t too bad. Enter your array by creating two text files arrayA.txt and arrayB.txt. One number per line. The source code for sort.pas is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
program Sort;

var
        A: array[1..10] of Integer;
        B: array[1..20] of Integer;
        F: Text;
        i,j,k,l,m,temp: Integer;

begin
        {Read in array A}
        Assign(F, 'arrayA.txt');
        Reset(F);
        i:= 0;
        while not EOF(F) do begin
                Inc(i);
                Read(F, A[i]);
        end;

        {Read in array B}
        Assign(F, 'arrayB.txt');
        Reset(F);
        j:= 0;
        while not EOF(F) do begin
                Inc(j);
                Read(F, B[j]);
        end;
        i:=10;
        j:=20;

        {Print out the unsorted arrays}
        WriteLn('Unsorted Arrays:');
        WriteLn('Array A:');
        for k:=1 to i do
        Write(A[k], ' ');
        WriteLn();
        WriteLn('Array B:');
        for k:=1 to j do
        Write(B[k], ' ');
        WriteLn();
        WriteLn('=========================');
        WriteLn('Sorting Arrays...');
        WriteLn('=========================');

        {Selection Sort Array A}
        for l := 1 to i do
                for m := l + 1 to i do
                        if A[l] > A[m] then
                        begin
                                temp := A[l];
                                A[l] := A[m];
                                A[m] := temp;
                        end;
        {Selection Sort Array B}
        for l := 1 to j do
                for m := l + 1 to j do
                        if B[l] > B[m] then
                        begin
                                temp := B[l];
                                B[l] := B[m];
                                B[m] := temp;
                        end;

        {Print out the sorted arrays}
        WriteLn('Selection Sorted Arrays:');
        WriteLn('Array A: ');
        for k:=1 to i do
        Write(A[k], ' ');
        WriteLn();
        WriteLn('Array B: ');
        for k:=1 to j do
        Write(B[k], ' ');
        WriteLn();
end.

Compile and run (ok to ignore the compile-time errors)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
dave@cerberus:~/Pascal$ fpc sort.pas
Free Pascal Compiler version 2.4.0-2 [2010/03/06] for x86_64
Copyright (c) 1993-2009 by Florian Klaempfl
Target OS: Linux for x86-64
Compiling sort.pas
Linking sort
/usr/bin/ld: warning: link.res contains output sections; did you forget -T?
73 lines compiled, 0.1 sec
dave@cerberus:~/Pascal$ ./sort
Unsorted Arrays:
Array A:
28 24 85 55 43 6 23 13 59 71
Array B:
13 37 36 53 24 83 27 42 62 71 9 92 1 41 6 3 88 77 65 67
=========================
Sorting Arrays...
=========================
Selection Sorted Arrays:
Array A:
6 13 23 24 28 43 55 59 71 85
Array B:
1 3 6 9 13 24 27 36 37 41 42 53 62 65 67 71 77 83 88 92
dave@cerberus:~/Pascal$

And there you have it. Compiling Pascal program on Ubuntu is an easy way to get your feet wet in programming. Pascal is a great beginner’s programming language, but if you want to learn more there are a number of great resources available for learning Pascal.


Copy Files to Many USB Drives Quickly and Easily

Posted 29th September in Hardware, Other Code, Programs. 1 Comment

photo

I was recently tasked with copying speaker’s presentations, files and handouts onto 100s of USB Drives (key drives) for a conference that work is hosting down in Washington, D.C. My first thought was that it was going to be a pain to have to copy/paste the files to each drive. I thought about creating a batch script to copy the files with a double click. But really, who wants to be doing all of that clicking and/or typing? Work smarter, not harder.

Then I remembered a neat feature that SyncBackSE, a program I use at home for backups, has available. The backup program – which is basically a file copy process – can be triggered based on the insert of a drive, whether that be a USB Key Drive or an External Hard Drive. Using the program, the only action you need to do to trigger the copy process is literally plug the drive in. After the machine recognizes the drive and mounts it to a drive letter, the copy process starts automatically.

Here are the directions on how to set this up.

  1. USB Hub

    Prepare your system. Install SyncBackSE. I have purchased the full program at home, but since I am using this for a limited task you can just install the 30-day trial. Create a folder, on your desktop or location of choice, with the files you want to be copied to the filesystem. Copy files to that folder, this is your “source directory”. Plug in as many USB drives as you can, in my case I used a USB hub with two handy top USB ports. Note which letter these drives assign themselves as.

  2. Set up the Source and Destination. The Source will be the “source directory” of any files you wish to copy. The Destination will be the USB Drive Letter. If you are able to plug more than one drive in at a time, you will need to create separate profiles for each one, with differing drive letters. Create them at first or just copy the first profile you make, and switch the Destination Drive on each one.
  3. Source and Destination Settings

  4. Modify the profile. This is where the magic happens. Under profile setup, go to “Click for Options” and check off “Expert”. Select When and then “Insert”. Check off “Run this profile when:” and select the drive letter under “is inserted into drive”. Also check off “Run unattended” – this will ensure it does not prompt you when you plug in your drive each time. The other options should remain with . A summary of these settings is in the figure below.

Settings for Auto Copy

That is it! Test by inserting a USB drive. You should see the profile change to “Running” then then “Success” after files have been completed. The screen will look as follows:

Final Product

In my experience, the part that takes the longest was Windows XP discovering the new drive and then assigning the drive letter. This PC is slow, so I imagine on a faster machine this process would actually be much quicker. I ended up having to re-copy the drives since someone added their handouts in the last minute, but the drive discovery process happened much faster the second time around. I was able to do about 100 drives in 30 minutes, so this process is actually very speedy and works very well.


How to Install SNMP on Tomato Router Firmware and Graph Traffic with Cacti

You’ve flashed your old WRT54G or other vanilla router with the Tomato firmware. This itself turns your router into a lean, mean routing machine with QOS, SSH and more, but let’s say we want to take it a bit further. What it we want to get some more stats out of it?

In order to do this, we first need to set up a way to pull this information from the router. The best way to do this is to install an SNMP (Simple Network Management Protocol) daemon on the system.

The main roadblock we face here is that the system mainly runs in volatile system memory, meaning that every time the system is rebooted the filesystem is reset. Fortunately Tomato provides a way to get around this using CIFS shares. Follow the steps below (as modified from here) to install an SNMP server on a Tomato router.

  1. Create a network (samba, CIFS) share somewhere on the network. This computer must be on all of the time in order for Tomato to run the SNMP server.
  2. Download the snmpd.zip file from one of these locations:
    [xs4all.nl]
    [systembash]

    expand the binary and .conf file into the share or a subdirectory (for example, <share name>/snmp)

    MD5 for snmpd binary is ae0d622648efdb8dceb7b3b5a63e23ac

  3. Set up the shared directory on the router. Visit Administration->CIFS Client. Add the share as follows, with your correct share information:cifs1
  4. Log into the Tomato router via ssh, and start SNMPd on the router by issuing the command:
    /cifs1/snmp/snmpd -c /cifs1/snmp/snmpd.conf &
  5. Test that SNMP is running and can be accessed on another computer on the network. To test it, you can use snmpwalk like so:
    snmpwalk -c public -v 2c <IP Address of Router>

    If it works properly, it will list the available OIDs from the router. You do not need to take note of these, but they will be used in the graphing software later.

  6. Finally, we need to launch the SNMP server when the router is restarted. You do this by adding the command to start it in the area Administration -> Scripts -> Firewall:
    sleep 30
    /cifs1/snmp/snmpd -c /cifs1/snmp/snmpd.conf -s &

    This launches the snmp server 30 seconds after the router is started or rebooted.

Thats it! SNMP is now running on the router.

Now to add this SNMP host to your graphing software. For this example, I will use Cacti, which I will assume you have already set up. If you need to set it up, please follow the directions on the Cacti site for installation.

First, add the router as a new device, using the information below (change IP to suite your needs):

adddevice

After adding the device, you have several options depending on what sort of data you are looking for. For system information on the router – for example CPU usage, memory usage, etc; you can go directly to Create -> New Graphs. Select your device and then add the graph you are looking for.

The graph will show as a broken image at first, or a blank graph with “NaN” as the data source. Give it a few minutes to update, and the information should start to flow through. The ucd/net options work best, but feel free to experiment.

To get traffic stats on the interface, you first need to “Walk” the device.  Go back to your device list, and edit the device you added. Under “Associated Data Queries”, Add Data Query, add “SNMP – Interface Statistics” with Re-Index period as “Uptime goes backwards”. After adding it you should see under status something like: Success [39 Items, 6 Rows].

Since these data sources are now added, you can go back to Add a new Graph. After selecting the device, you should see a list of these new interfaces. Select the interfaces you wish to graph, and select the graph type (I suggest In/Out bits with Total).

After a few minutes, the data should start filling in. After a while, you will get a graph like this:

graph_image.php

In conclusion, with a little work, you can get enterprise class graphing from your consumer router. The total project took me about 45 minutes, and I was trying to figure out all of the data sources and the correct way to enter everything.

Let me know your experiences, suggestions and corrections!


Easy Search and Replace in Multiple Files on Linux Command Line

I recently came across a typo that existed in a bunch of html files on my web server. I thought it should be easy enough to change, but since it was in a number of files, editing it by hand would be time consuming. Fortunately, there is an easy, one liner command to replace the text in multiple files in a sub directory using recursion.

grep -lr -e '<oldword>' * | xargs sed -i 's/<oldword>/<newword>/g'

This command broken down:

  • grep for the word in a files, use recursion (to find files in sub directories), and list only file matches
  • | xargs passes the results from the grep command to sed
  • sed -i uses a regular expression (regex) to evaluate the change: s (search) / search word / target word / g (global replace)

For more information, see man pages for grep, sed, and xarg. Also it is very handy to learn about regular expressions as they are a valuable tool to any command line programmer!

Update 2009/7/19:

Thanks to reader btr we have a great one-line perl command that will perform the same task:

Perl provides a really nice one-line for this kind of thing:

1
perl -p -i -e ’s///g’ *

It also provides the option of creating a backup of each file changed:

1
perl -p -i.bak -e ’s///g’ *

mnemonic: PIE (”easy as pie”, etc.)
google “perl pie” and you’ll get lots of info for other uses of this technique.

http://www.linux.org/lessons/short/perlpie/perl_pie.html


Password Protected Folder Gives 404 Not Found in WordPress Installation Sub-folder.

Wordpress

Came across this little maddening issue again today after fixing it a few months back. I created a directory that is password protected using a .htaccess file. However, when trying to access this folder or anything under this directory, a File Not Found 404 error from WordPress is displayed before it even prompts you for the password. The problem here lays within the main WordPress .htaccess file

The default .htaccess file for WordPress is:

1
2
3
4
5
6
7
8
9
# BEGIN wordpress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END wordpress

This means that if a file is requested from the server, if it is not a file that exists in the server’s folder directory (!-f) and if it is not a directory that exists in the server’s folder directories (!-d) then pass the request onto index.php. This way, WordPress will handle both customized URLs (such as those used for SEO) and also 404 File Not Found errors.

If you set up a password protected folder in a directory included in a WordPress install, all of a sudden WordPress takes over that folder and returns a 404 page, like the file doesn’t exist.

This happens because of a little ‘gotcha’ in the apache configuration. Luckily it is an easy fix.

In the password protected folder’s .htaccess file, you may already have the entries to ask for password access. Before all of that, place the following line:

1
ErrorDocument 401 /401.html

Then create a 401.html in the main folder, with any text, for example:

1
PASSWORD PROTECTED FOLDER - Please enter the correct username/password.

Voila, you can now enter your password protected folder again.

There is another workaround this little error, but since it involves editing the main wordpress .htaccess file, it can be overridden during an upgrade.

Solution found on: http://www.webmasterworld.com/apache/3688208.htm