Posts

Centos pip python install error

While attempting to install Thumbor on a CentOS server I recently had the following error message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
------------------------------------------------------------
/usr/bin/pip run on Thu Sep 18 21:07:45 2014
Getting page https://pypi.python.org/simple/pycrypto/
URLs to search for versions for pycrypto in /usr/lib64/python2.6/site-packages:
* https://pypi.python.org/simple/pycrypto/
Analyzing links from page https://pypi.python.org/simple/pycrypto/
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.0.1.tar.gz#md5=4d5674f3898a573691ffb335e8d749cd (from https://pypi.python.org/simple/pycrypto/), version: 2.0.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.1.0.tar.gz#md5=1d3eb04f06e6f09a080bc37fb019f9bf (from https://pypi.python.org/simple/pycrypto/), version: 2.1.0
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.2.tar.gz#md5=4f0ed728b14b98f09120cb2ec461ec98 (from https://pypi.python.org/simple/pycrypto/), version: 2.2
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.3.tar.gz#md5=2b811cfbfc342d83ee614097effb8101 (from https://pypi.python.org/simple/pycrypto/), version: 2.3
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.1.tar.gz#md5=c2a1404a848797fb0806f3e11c29ef15 (from https://pypi.python.org/simple/pycrypto/), version: 2.4.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.tar.gz#md5=274fa44c30a320d56460a93fdd95e702 (from https://pypi.python.org/simple/pycrypto/), version: 2.4
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz#md5=783e45d4a1a309e03ab378b00f97b291 (from https://pypi.python.org/simple/pycrypto/), version: 2.5
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda (from https://pypi.python.org/simple/pycrypto/), version: 2.6.1
  Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.tar.gz#md5=88dad0a270d1fe83a39e0467a66a22bb (from https://pypi.python.org/simple/pycrypto/), version: 2.6
Using version 2.6.1 (newest of versions: 2.6.1, 2.6, 2.5, 2.4.1, 2.4, 2.3, 2.2, 2.1.0, 2.0.1, 2.0.1)
Downloading/unpacking pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda

  Running setup.py egg_info for package pycrypto

    running egg_info
    writing pip-egg-info/pycrypto.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pycrypto.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pycrypto.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found
    reading manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/pycrypto.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-root/pycrypto has version 2.6.1, which satisfies requirement pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda
Installing collected packages: pycrypto

  Found existing installation: pycrypto 2.0.1

    Uninstalling pycrypto:

      Removing file or directory /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
      Successfully uninstalled pycrypto

  Running setup.py install for pycrypto

    Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed
    running install
    running build
    running build_py
    running build_ext
    running build_configure
    checking for gcc... gcc

    checking whether the C compiler works... yes

    checking for C compiler default output file name... a.out

    checking for suffix of executables...

    checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

    configure: error: cannot run C compiled programs.

    If you meant to cross compile, use `--host'.

    See `config.log' for more details

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

        core.setup(**kw)

      File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

        dist.run_commands()

      File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

        self.run_command(cmd)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

        return _install.run(self)

      File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

        self.run_command('build')

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

        self.run_command(cmd_name)

      File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

        self.distribution.run_command(command)

      File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

        cmd_obj.run()

      File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

        raise RuntimeError("autoconf error")

    RuntimeError: autoconf error

    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_py

running build_ext

running build_configure

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `--host'.

See `config.log' for more details

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-root/pycrypto/setup.py", line 456, in <module>

    core.setup(**kw)

  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands

    self.run_command(cmd)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib/python2.6/site-packages/setuptools/command/install.py", line 53, in run

    return _install.run(self)

  File "/usr/lib64/python2.6/distutils/command/install.py", line 577, in run

    self.run_command('build')

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/usr/lib64/python2.6/distutils/command/build.py", line 134, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 251, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command

    cmd_obj.run()

  File "/tmp/pip-build-root/pycrypto/setup.py", line 278, in run

    raise RuntimeError("autoconf error")

RuntimeError: autoconf error

----------------------------------------

  Rolling back uninstall of pycrypto

  Replacing /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/usr/lib/python2.6/site-packages/pip/commands/install.py", line 271, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 1185, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pip/req.py", line 592, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/pycrypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-C4u4v3-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

It essentially boils down to:

1
2
3
4
checking whether we are cross compiling... configure: error: in `/tmp/pip-build-root/pycrypto':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

Weird, I have gcc and all compile programs installed.

It took me a fair time of troubleshooting, but I finally figured out it was because it was attempting to build this in /tmp, which I have set to mount at noexec for security purposes. This disallows execution of programs in this directory.

Running

1
mount -oremount,exec /tmp

Allowed it to run without issue.

A Poor Man’s VPN: Proxy Web Connection to Remote Server (via SSH and Tunnel)

Did you ever have a situation where you needed to access a website that had an IP restriction in place? I recently had a situation where I needed to access the web via my university connection (due to IP restrictions placed on accessing databases of research papers). They do not have a VPN setup so it is hard to do this off-campus.

I do however have access to a linux machine on campus. I am familiar with port forwarding using SSH but I had never used it to actually tunnel web traffic using a web browser on Windows. Turns out it is surprisingly easy!

The ssh command to use is:

1
ssh -C2qTnN -D 8080 username@remote_host

This command sshes to the remote_host, and creates a tunnel on your localhost, port 8080. Note that you need to have private key authentication already set up for this host – it will not work with password authentication.

The description of the switches are (from the ssh man page):

  • -C : Compression
  • -2 : Use SSHv2
  • -q : quiet!
  • -T : Disable pseuto-tty allocation
  • -n : Prevents reading from stdin (you need to have private key authentication set up, to prevent password authentication)
  • -N : Do not execute a remote command (or launch a shell). Just use the ssh process for port forwarding
  • -D : Allocate a socket to listen on the local side. When a connection is made to this port it is located to the remote machine. Makes SSH work as a SOCKS server. Only root can forward privileged ports like this.

From here, you set up Firefox or your browser of choice to use a Socks proxy on localhost:8080. The man page says that SOCKS4 and SOCK5 should both work but I had to use SOCKS v4, SOCKS v5 did not seem to work for me.

Bash Script to Interface with Rsync Command

I created this Bash script as a project for the system administration course I’m taking for the summer. I’m sure there are bugs in it, so let me know if you find any.

It basically uses an XML configuration file, which includes the source, destination, and any excludes from the transfer. You then pass either -u or -d (upload or download) and the options -x (delete if not in source) and -f (force). Destination can be local or remote, but source must be local.

Here is the code:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
#!/bin/bash
## Transfer Script
## By David Drager
## CSC586: Summer II 2008
## Requires: xml2, rsync

## Settings
tempdir="/tmp/"
rsynccommand="/usr/bin/rsync"

## Usage command
usage="usage: transfer.sh [options] <config-file>.xmlnoptions:n  -d or -u:t download or upload, resp. one and only one must be presentn  -f:ttforce transfer regardless of 'newness' of filen  -x:ttdelete items in target not present in source"


# Check to make sure temporary directory exists
[ -d $tempdir ] || { echo -e "Error: Could not locate a temporary directory. See file settings."; exit 1; }

# Usage Command
[ $# -eq 0 ] && { echo -e "ERROR: Needs at least the -d or -u flag plus a config file.nn$usage"; exit 1; }


## Get options passed to the program
while getopts "dufx" flag
do
  [ "$flag" = "?" ] && echo -e "ERROR: Flag(s) not valid.nn$usage" && exit 1;
  eval "opt_$flag=1"
done
shift $((OPTIND-1))
configfile="$1"

if [ "$opt_d" = 1 ] && [ "$opt_u" = 1 ]
then
  echo -e "Error: Use either -u Upload or -d Download, but not bothn$usage"; exit 1;
elif  [ "$opt_u" = "" ] && [ "$opt_d" = "" ]
then
    echo -e "Error: Use either -u Upload or -d Download, but not bothn$usage"; exit 1;
fi

if [ "$opt_f" = 1 ]; then
  force=1
fi

if [ "$opt_x" = 1 ]; then
  deltar=1
fi

## Make sure config file is located
if [ "$configfile" != "" ]
then
  [ -f "$configfile" ] || { echo -e "Error: Config file not found.n$usage"; exit 1; }
else
   echo -e "Error: Config file not specified.n$usage"; exit 1;
fi

## End of swich verification

# echo Action: "$action"
# echo Force: "$force"
# echo Delete Target Files: "$deltar"
# echo Config: "$configfile"

## Now read config file

## Set temp file for XML parsing
tempfile="$tempdir"transferscript_$$
tempexcludes="$tempdir"transferscriptexcludes_$$
# echo -e "Temp:$tempfile"

# Run config file through xml2
xml2 < "$configfile" > "$tempfile"

[ -f "$tempfile" ] || { echo -e "There was a problem processing the XML file."; exit 1; }

# Make sure that source and destination are found
excludelist=""
while read line; do
    key=$(echo $line | awk -F '=' '{print $1}')
    value=$(echo $line | awk -F '=' '{print $2}')
    if [ "$key" = "/sync/@src" ]; then
      source="$value";
    elif [ "$key" = "/sync/@dst" ]; then
      destination="$value"
      # Add any exclude options to array
    elif [ "$key" = "/sync/exclude" ]; then
      tempexcl=("$value")
      excludelist=("${excludelist[@]}" "${tempexcl[@]}")
    fi
done < "$tempfile"

[ "$source" ] || { echo "Error: Config file must include a source"; exit 1; }
[ "$destination" ] || { echo "Error: Config file must include a destination"; exit 1; }

source="$source/"
destination="$destination/"

[ -d "$source" ] || { echo "Error: Source must be a local directory"; exit 1; }

excludes=""
excludes=${excludelist[@]}

# echo Excludes: "(${#excludelist[@]}): $excludes"

## Clean up temp file from xml process
rm "$tempfile"



## Build up the command line

[ -f "$rsynccommand" ] || { echo -e "Error: Could not locate rsync command."; exit 1; }

fullcommand="$rsynccommand"
fulltestcommand="$rsynccommand"

if [ "$opt_f" == "1" ]
then
    fulltestcommand="$fulltestcommand -nva"
    fullcommand="$fullcommand -a"
else
    fulltestcommand="$fulltestcommand -nvau"
    fullcommand="$fullcommand -au"
fi

if [ "$opt_u" == "1" ]
then
  fileorder="$source $destination"
  echo "Evaluating transfer from $source ==> $destination"
elif [ "$opt_d" == "1" ]
then
  fileorder="$destination $source"
  echo "Evaluating transfer from $destination ==> $source"
fi

[ "$opt_x" == "1" ] && { fulltestcommand="$fulltestcommand --delete"; fullcommand="$fullcommand --delete"; }


## Move exclude list to a file.
## This duplicates as above but I originally thought we were passing it to rsync as a list and not as a file.

[ "${#excludelist[@]}" -gt 0 ] && {
    tempfile="$tempdir"transferscript_$$
    touch "$tempfile"
    echo "$excludes" | tr " " "n" > "$tempfile"
    fulltestcommand="$fulltestcommand"" --exclude-from=$tempfile"
    fullcommand="$fullcommand"" --exclude-from=$tempfile"
}

fulltestcommand="$fulltestcommand $fileorder"
fullcommand="$fullcommand $fileorder"

# We only want lines minus top line and bottom 3 lines
testresult=`$fulltestcommand | tail --lines=+2 | head --lines=-3`

# If this is empty, then notify that no files would be transferred.
 [ "$testresult" ] || { echo "This command would not transfer any files. Script exiting."; rm "$tempfile"; exit 1; }

# Display changes
echo -e "Changes to be made:n--------------------n$testresultn--------------------"

# Check to see if we really want to run the command.
echo "Do you wish to perform this command? $fullcommand (y/n)[y]"
read confirm
if [ "$confirm" = "y" ] || [ "$confirm" = "" ]
then
    echo "Performing transfer..."
    # Without -v this will not output anything.
    $fullcommand
    echo "Transfer complete."
    rm "$tempfile"
else
    echo "Cancelling transfer..."
    rm "$tempfile"
fi

exit 0

Here is the sample config file:


1
2
3
4
5
6
<sync src="/home/dir/bash-prog/dir1/" dst="login@some.machine:/home/dir/dir2">
        <exclude>
          .test3
          another.*
        </exclude>
</sync>