Posts

Centos pip python install error

While attempting to install Thumbor on a CentOS server I recently had the following error message:

[cc lines=”40″]————————————————————
/usr/bin/pip run on Thu Sep 18 21:07:45 2014
Getting page https://pypi.python.org/simple/pycrypto/
URLs to search for versions for pycrypto in /usr/lib64/python2.6/site-packages:
* https://pypi.python.org/simple/pycrypto/
Analyzing links from page https://pypi.python.org/simple/pycrypto/
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.0.1.tar.gz#md5=4d5674f3898a573691ffb335e8d749cd (from https://pypi.python.org/simple/pycrypto/), version: 2.0.1
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.1.0.tar.gz#md5=1d3eb04f06e6f09a080bc37fb019f9bf (from https://pypi.python.org/simple/pycrypto/), version: 2.1.0
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.2.tar.gz#md5=4f0ed728b14b98f09120cb2ec461ec98 (from https://pypi.python.org/simple/pycrypto/), version: 2.2
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.3.tar.gz#md5=2b811cfbfc342d83ee614097effb8101 (from https://pypi.python.org/simple/pycrypto/), version: 2.3
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.1.tar.gz#md5=c2a1404a848797fb0806f3e11c29ef15 (from https://pypi.python.org/simple/pycrypto/), version: 2.4.1
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.4.tar.gz#md5=274fa44c30a320d56460a93fdd95e702 (from https://pypi.python.org/simple/pycrypto/), version: 2.4
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz#md5=783e45d4a1a309e03ab378b00f97b291 (from https://pypi.python.org/simple/pycrypto/), version: 2.5
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda (from https://pypi.python.org/simple/pycrypto/), version: 2.6.1
Found link https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.tar.gz#md5=88dad0a270d1fe83a39e0467a66a22bb (from https://pypi.python.org/simple/pycrypto/), version: 2.6
Using version 2.6.1 (newest of versions: 2.6.1, 2.6, 2.5, 2.4.1, 2.4, 2.3, 2.2, 2.1.0, 2.0.1, 2.0.1)
Downloading/unpacking pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda

Running setup.py egg_info for package pycrypto

running egg_info
writing pip-egg-info/pycrypto.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pycrypto.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pycrypto.egg-info/dependency_links.txt
warning: manifest_maker: standard file ‘-c’ not found
reading manifest file ‘pip-egg-info/pycrypto.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘pip-egg-info/pycrypto.egg-info/SOURCES.txt’
Source in /tmp/pip-build-root/pycrypto has version 2.6.1, which satisfies requirement pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.1.tar.gz#md5=55a61a054aa66812daf5161a0d5d7eda
Installing collected packages: pycrypto

Found existing installation: pycrypto 2.0.1

Uninstalling pycrypto:

Removing file or directory /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Successfully uninstalled pycrypto

Running setup.py install for pycrypto

Running command /usr/bin/python -c “import setuptools;__file__=’/tmp/pip-build-root/pycrypto/setup.py’;exec(compile(open(__file__).read().replace(‘\r\n’, ‘\n’), __file__, ‘exec’))” install –record /tmp/pip-C4u4v3-record/install-record.txt –single-version-externally-managed
running install
running build
running build_py
running build_ext
running build_configure
checking for gcc… gcc

checking whether the C compiler works… yes

checking for C compiler default output file name… a.out

checking for suffix of executables…

checking whether we are cross compiling… configure: error: in `/tmp/pip-build-root/pycrypto’:

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `–host’.

See `config.log’ for more details

Traceback (most recent call last):

File ““, line 1, in

File “/tmp/pip-build-root/pycrypto/setup.py”, line 456, in

core.setup(**kw)

File “/usr/lib64/python2.6/distutils/core.py”, line 152, in setup

dist.run_commands()

File “/usr/lib64/python2.6/distutils/dist.py”, line 975, in run_commands

self.run_command(cmd)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/usr/lib/python2.6/site-packages/setuptools/command/install.py”, line 53, in run

return _install.run(self)

File “/usr/lib64/python2.6/distutils/command/install.py”, line 577, in run

self.run_command(‘build’)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/usr/lib64/python2.6/distutils/command/build.py”, line 134, in run

self.run_command(cmd_name)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/tmp/pip-build-root/pycrypto/setup.py”, line 251, in run

self.run_command(cmd_name)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/tmp/pip-build-root/pycrypto/setup.py”, line 278, in run

raise RuntimeError(“autoconf error”)

RuntimeError: autoconf error

Complete output from command /usr/bin/python -c “import setuptools;__file__=’/tmp/pip-build-root/pycrypto/setup.py’;exec(compile(open(__file__).read().replace(‘\r\n’, ‘\n’), __file__, ‘exec’))” install –record /tmp/pip-C4u4v3-record/install-record.txt –single-version-externally-managed:

running install

running build

running build_py

running build_ext

running build_configure

checking for gcc… gcc

checking whether the C compiler works… yes

checking for C compiler default output file name… a.out

checking for suffix of executables…

checking whether we are cross compiling… configure: error: in `/tmp/pip-build-root/pycrypto’:

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `–host’.

See `config.log’ for more details

Traceback (most recent call last):

File ““, line 1, in

File “/tmp/pip-build-root/pycrypto/setup.py”, line 456, in

core.setup(**kw)

File “/usr/lib64/python2.6/distutils/core.py”, line 152, in setup

dist.run_commands()

File “/usr/lib64/python2.6/distutils/dist.py”, line 975, in run_commands

self.run_command(cmd)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/usr/lib/python2.6/site-packages/setuptools/command/install.py”, line 53, in run

return _install.run(self)

File “/usr/lib64/python2.6/distutils/command/install.py”, line 577, in run

self.run_command(‘build’)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/usr/lib64/python2.6/distutils/command/build.py”, line 134, in run

self.run_command(cmd_name)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/tmp/pip-build-root/pycrypto/setup.py”, line 251, in run

self.run_command(cmd_name)

File “/usr/lib64/python2.6/distutils/cmd.py”, line 333, in run_command

self.distribution.run_command(command)

File “/usr/lib64/python2.6/distutils/dist.py”, line 995, in run_command

cmd_obj.run()

File “/tmp/pip-build-root/pycrypto/setup.py”, line 278, in run

raise RuntimeError(“autoconf error”)

RuntimeError: autoconf error

—————————————-

Rolling back uninstall of pycrypto

Replacing /usr/lib64/python2.6/site-packages/pycrypto-2.0.1-py2.6.egg-info
Command /usr/bin/python -c “import setuptools;__file__=’/tmp/pip-build-root/pycrypto/setup.py’;exec(compile(open(__file__).read().replace(‘\r\n’, ‘\n’), __file__, ‘exec’))” install –record /tmp/pip-C4u4v3-record/install-record.txt –single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto

Exception information:
Traceback (most recent call last):
File “/usr/lib/python2.6/site-packages/pip/basecommand.py”, line 139, in main
status = self.run(options, args)
File “/usr/lib/python2.6/site-packages/pip/commands/install.py”, line 271, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File “/usr/lib/python2.6/site-packages/pip/req.py”, line 1185, in install
requirement.install(install_options, global_options, *args, **kwargs)
File “/usr/lib/python2.6/site-packages/pip/req.py”, line 592, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File “/usr/lib/python2.6/site-packages/pip/util.py”, line 662, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c “import setuptools;__file__=’/tmp/pip-build-root/pycrypto/setup.py’;exec(compile(open(__file__).read().replace(‘\r\n’, ‘\n’), __file__, ‘exec’))” install –record /tmp/pip-C4u4v3-record/install-record.txt –single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/pycrypto[/cc]

It essentially boils down to:

[cc]checking whether we are cross compiling… configure: error: in `/tmp/pip-build-root/pycrypto’:
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `–host’.
See `config.log’ for more details[/cc]

Weird, I have gcc and all compile programs installed.

It took me a fair time of troubleshooting, but I finally figured out it was because it was attempting to build this in /tmp, which I have set to mount at noexec for security purposes. This disallows execution of programs in this directory.

Running

[cc]mount -oremount,exec /tmp[/cc]

Allowed it to run without issue.

A Poor Man’s VPN: Proxy Web Connection to Remote Server (via SSH and Tunnel)

Did you ever have a situation where you needed to access a website that had an IP restriction in place? I recently had a situation where I needed to access the web via my university connection (due to IP restrictions placed on accessing databases of research papers). They do not have a VPN setup so it is hard to do this off-campus.

I do however have access to a linux machine on campus. I am familiar with port forwarding using SSH but I had never used it to actually tunnel web traffic using a web browser on Windows. Turns out it is surprisingly easy!

The ssh command to use is:

ssh -C2qTnN -D 8080 username@remote_host

This command sshes to the remote_host, and creates a tunnel on your localhost, port 8080. Note that you need to have private key authentication already set up for this host – it will not work with password authentication.

The description of the switches are (from the ssh man page):

  • -C : Compression
  • -2 : Use SSHv2
  • -q : quiet!
  • -T : Disable pseuto-tty allocation
  • -n : Prevents reading from stdin (you need to have private key authentication set up, to prevent password authentication)
  • -N : Do not execute a remote command (or launch a shell). Just use the ssh process for port forwarding
  • -D : Allocate a socket to listen on the local side. When a connection is made to this port it is located to the remote machine. Makes SSH work as a SOCKS server. Only root can forward privileged ports like this.

From here, you set up Firefox or your browser of choice to use a Socks proxy on localhost:8080. The man page says that SOCKS4 and SOCK5 should both work but I had to use SOCKS v4, SOCKS v5 did not seem to work for me.

Bash Script to Interface with Rsync Command

I created this Bash script as a project for the system administration course I’m taking for the summer. I’m sure there are bugs in it, so let me know if you find any.

It basically uses an XML configuration file, which includes the source, destination, and any excludes from the transfer. You then pass either -u or -d (upload or download) and the options -x (delete if not in source) and -f (force). Destination can be local or remote, but source must be local.

Here is the code:

#!/bin/bash
## Transfer Script
## By David Drager
## CSC586: Summer II 2008
## Requires: xml2, rsync

## Settings
tempdir="/tmp/"
rsynccommand="/usr/bin/rsync"

## Usage command
usage="usage: transfer.sh [options] .xmlnoptions:n  -d or -u:t download or upload, resp. one and only one must be presentn  -f:ttforce transfer regardless of 'newness' of filen  -x:ttdelete items in target not present in source"


# Check to make sure temporary directory exists
[ -d $tempdir ] || { echo -e "Error: Could not locate a temporary directory. See file settings."; exit 1; }

# Usage Command
[ $# -eq 0 ] && { echo -e "ERROR: Needs at least the -d or -u flag plus a config file.nn$usage"; exit 1; }


## Get options passed to the program
while getopts "dufx" flag
do
  [ "$flag" = "?" ] && echo -e "ERROR: Flag(s) not valid.nn$usage" && exit 1;
  eval "opt_$flag=1"
done
shift $((OPTIND-1))
configfile="$1"

if [ "$opt_d" = 1 ] && [ "$opt_u" = 1 ]
then
  echo -e "Error: Use either -u Upload or -d Download, but not bothn$usage"; exit 1;
elif  [ "$opt_u" = "" ] && [ "$opt_d" = "" ]
then
    echo -e "Error: Use either -u Upload or -d Download, but not bothn$usage"; exit 1;
fi

if [ "$opt_f" = 1 ]; then
  force=1
fi

if [ "$opt_x" = 1 ]; then
  deltar=1
fi

## Make sure config file is located
if [ "$configfile" != "" ]
then
  [ -f "$configfile" ] || { echo -e "Error: Config file not found.n$usage"; exit 1; }
else
   echo -e "Error: Config file not specified.n$usage"; exit 1;
fi

## End of swich verification

# echo Action: "$action"
# echo Force: "$force"
# echo Delete Target Files: "$deltar"
# echo Config: "$configfile"

## Now read config file

## Set temp file for XML parsing
tempfile="$tempdir"transferscript_$$
tempexcludes="$tempdir"transferscriptexcludes_$$
# echo -e "Temp:$tempfile"

# Run config file through xml2
xml2 < "$configfile" > "$tempfile"

[ -f "$tempfile" ] || { echo -e "There was a problem processing the XML file."; exit 1; }

# Make sure that source and destination are found
excludelist=""
while read line; do
    key=$(echo $line | awk -F '=' '{print $1}')
    value=$(echo $line | awk -F '=' '{print $2}')
    if [ "$key" = "/sync/@src" ]; then
      source="$value";
    elif [ "$key" = "/sync/@dst" ]; then
      destination="$value"
      # Add any exclude options to array
    elif [ "$key" = "/sync/exclude" ]; then
      tempexcl=("$value")
      excludelist=("${excludelist[@]}" "${tempexcl[@]}")
    fi
done < "$tempfile"

[ "$source" ] || { echo "Error: Config file must include a source"; exit 1; }
[ "$destination" ] || { echo "Error: Config file must include a destination"; exit 1; }

source="$source/"
destination="$destination/"

[ -d "$source" ] || { echo "Error: Source must be a local directory"; exit 1; }

excludes=""
excludes=${excludelist[@]}

# echo Excludes: "(${#excludelist[@]}): $excludes"

## Clean up temp file from xml process
rm "$tempfile"



## Build up the command line

[ -f "$rsynccommand" ] || { echo -e "Error: Could not locate rsync command."; exit 1; }

fullcommand="$rsynccommand"
fulltestcommand="$rsynccommand"

if [ "$opt_f" == "1" ]
then
    fulltestcommand="$fulltestcommand -nva"
    fullcommand="$fullcommand -a"
else
    fulltestcommand="$fulltestcommand -nvau"
    fullcommand="$fullcommand -au"
fi

if [ "$opt_u" == "1" ]
then
  fileorder="$source $destination"
  echo "Evaluating transfer from $source ==> $destination"
elif [ "$opt_d" == "1" ]
then
  fileorder="$destination $source"
  echo "Evaluating transfer from $destination ==> $source"
fi

[ "$opt_x" == "1" ] && { fulltestcommand="$fulltestcommand --delete"; fullcommand="$fullcommand --delete"; }


## Move exclude list to a file.
## This duplicates as above but I originally thought we were passing it to rsync as a list and not as a file.

[ "${#excludelist[@]}" -gt 0 ] && {
    tempfile="$tempdir"transferscript_$$
    touch "$tempfile"
    echo "$excludes" | tr " " "n" > "$tempfile"
    fulltestcommand="$fulltestcommand"" --exclude-from=$tempfile"
    fullcommand="$fullcommand"" --exclude-from=$tempfile"
}

fulltestcommand="$fulltestcommand $fileorder"
fullcommand="$fullcommand $fileorder"

# We only want lines minus top line and bottom 3 lines
testresult=`$fulltestcommand | tail --lines=+2 | head --lines=-3`

# If this is empty, then notify that no files would be transferred.
 [ "$testresult" ] || { echo "This command would not transfer any files. Script exiting."; rm "$tempfile"; exit 1; }

# Display changes
echo -e "Changes to be made:n--------------------n$testresultn--------------------"

# Check to see if we really want to run the command.
echo "Do you wish to perform this command? $fullcommand (y/n)[y]"
read confirm
if [ "$confirm" = "y" ] || [ "$confirm" = "" ]
then
    echo "Performing transfer..."
    # Without -v this will not output anything.
    $fullcommand
    echo "Transfer complete."
    rm "$tempfile"
else
    echo "Cancelling transfer..."
    rm "$tempfile"
fi

exit 0

Here is the sample config file:


        
          .test3
          another.*