Python development on Windows [Part 2]: Installing easy_install...could be easier

Assuming you followed Part 1 sucessfully, you already have a pretty functional python environment. To insure you have all the pre-requisites open up a dos prompt and type "python", if you are greeted with the python prompt, you are good to go, if not, go review part 1, we'll wait.

In this part of the series we'll be setting up easy_install. Why should you care about easy_install? Well, because python, like most modern programming languages/frameworks get's a lot of it's functionality from libraries or packages. Thus, making it easier on you to write software because you can rely on functionality that other people have written. And there are some great packages out there to explore.

But relying on libraries presents a problem. For one, you have to install the libraries before you use them. And there's a good chance that those libraries depend on other libraries which you or may not have installed. And they may depend on different versions. Managing dependencies between libraries can quickly become a full-time job, thankfully tools have been developed to specifically manage this task so that your job is easier. 

One tool for managing libraries in python is easy_install, and most libraries I've come across can be had through this tool.

Before we begin, I want to note that alot of this stuff is likely applicable to any operating system that python can run on. So why am I making it specific to Windows?

Well the answer is, because there are some differences in how to set it up on windows versus other OS's, probably because Windows doesn't quite share the same lineage as other OS's, therefore, in effect, what we're ultimately doing here is making Windows behave a bit more like those other OS's in regards to python.

OK, let's get started. First we'll need to download setuptools. As of right now the current version is 0.6c9 . Download the exe and run it. All of the defaults should be fine unless you did something fancy. And if you did, you likely don't need to read this.

The install is quick and painless. however, you aren't done yet.  To see why, open a command prompt and type "easy_install" and see what happens. The same old "... is not recognized as in internal or external command" message. What we want is for the program to actually run when we type easy_install. The way to make that happen is the same process we did for making python work in part 1. Which is to add whichever directory contains easy_install.

The directory that contains easy_install is by default "c:\Python25\Scripts" which seems to be a pretty typical place to drop tools like this as we will see in later tutorials. But since we already defined PYTHON_HOME = C:\Python25 all that is necessary is to append %PYTHON_HOME%\Scripts to the path. AND DON"T FORGET THE ";" !!!

As in part 1, you can verify that everything is on the up and up by opening a command prompt and typing "easy_install". If you did everything correctly you should see something like the following screenshot.

Yep, your eyes don't decieve you, that's an error, but it's an error from easy_install, so since we wanted to get the system to find easy_install, the fact that we see an error from easy_install is actually a good thing for the moment.

So now, let's use it to actually install something. I'm going to go out on a limb and assume that you are interested in incorporating unit testing into your python development at some point. Well, it just so happens that there are indeed libraries for that. The python standard library actually comes with one, but I find myself using "nose" most of the time. To install it using easy_install open up a command prompt and type "easy_install nose" .

easy_install will spring to life and fetch nose and install it for you from the internet. Upon sucessful completion you should be able to type "nosetests" and see some output like the following screenshot.

Voila, you have a brand new command which will run a unit test suite for you whenever you want.

OK, so we now have the ability to pull down and install any python library right? Not quite, here's where we come across another step you typically don't think about in those other OS's but you will likely run across it often in Windows.

But first a little background. One feature of python is it's ability to use code written in C via extensions. You may find you need to learn to write one someday. But even if you don't, you need to be somewhat aware of them because you are likely to run across a library that includes it's own C extensions. 

In order for easy_install to remain somewhat platform neutral, the C extensions will likely be distributed as source along with the python source. When this is the case, the C extension needs to be compiled with a C++ compiler. On most flavors of Linux and other OS's this likely not an issue and the library will build the C extension without issue. On Windows however there is likely no such C++ compiler. To illustrate this situation try "easy_install pycrypto" now.

As you can see from the output there's a problem. And the problem is the C extensions could not be built. Now, to me the mention of Visual Studio 2003 seems to be a benign error. I'm not savvy enough or rich enough to buy or know how to setup VS2k3's C++ compiler to work with easy_install. The easiest/cheapest way I've found is to go the MingW32 route. So the next step is to install MingW32.

To install MingW32, I download the "Automated MingW installer" from here on sourceforge, but if it no longer exists, go to http://mingw.org. And look around until you find it.

Once downloaded and run, you are presented with several screens that you can just click through until you get to "Choose Components".

By default minimal is selected, which installs no compilers whatsoever. What we want is the "g++ compiler" and "MingW Make", or if you are fealing really sassy just select "full" and forget about it. After clicking next you are presented with an install directory option. The default is fine, but if you change it, keep in mind you will need it for the next step because we need to add the location of all the compilers to the path. After clicking finish, the installer pulls down everything it needs to and sets things up as you requested.

When the install completes, you will have a c++ compiler that easy_install can be told to use. However if you aren't sensing a common theme yet, try typing "g++" at the command prompt. Obviously this is not good, but you know by now that you can fix this by adding it to the path. The path for the compliers is by default "C:\MinGW\bin", so go ahead and add it to the path. You'll know you're good to go when you can type "g++" and something actually happens.

Ok there's only one last thing we need to do before we can get pycrypto to install via easy_install. And that is by telling easy_install to use MingW instead of punting to a non-existent Visual Studio 2003 install.

This is done by creating a file called "distutils.cfg" at C:\Python25\Lib\distutils that contains following.

[build]
compiler=mingw32

If you did that correctly, and you now run "easy_install pycrypto" you should get a successful install of pycrypto. To check you can launch python and type...

C:\Documents and Settings\twillis>python
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import Crypto
>>>

 

If you get no error message, PyCrypto is installed!!!!

At this point you should have a decent chance at installing a good number of packages that you may come across.

However, I should mention that in some cases, those C extensions may need access to header files of some sort. When that happens, you may find it's easier to by-pass easy_install and instead go with a binary distribution of the library.

Depending on headers implies that those C extensions are expecting something else be installed already that you may not have, and those things may have their own dependencies and install programs.

OK, so hopefully this has successfully walked you through how to setup easy_install on your Windows box, along with a C compiler for it to use to build any C extensions the library might require. And hopefully the reasons why we set it up that way.

At this point, I would say your machine is setup at roughly the same level as what is reasonable to expect from other OS's in regards to python development. Suggestions are always welcome though.

--compiler option !

I'am not using easy_install but I have to compile extensions on a Windows Box (because a client of mine is under this OS), but the problems are shared. So, I didn't edit the distutils.cfg thinking that doing :
python setup.py build --compiler=mingw32
Then :
python setup.py install
Would be the same. The answers is no. It complains about the compiler. The build works but the install fails (and this is normal !). Editing the distutils.cfg fixed the problem.

I can not imagine how

I can not imagine how frustrating this must've been/is. But hey now you have it working yes?

awesome instructions

thank you so much for walking us all through this, awesome instructions!

hooh, wow, I'm f** glad I

hooh, wow, I'm f** glad I found this...shit I was stuck for a looooOoong time, you're the best!

windows sdk might be enough

Installing the windows sdk might be enough since it also includes the microsoft compiler suite

This was very helpful. thanx

This was very helpful. thanx for posting this, much needed. :-)

Thank you

Your instructions helped me to get a bdist_wininst of a few other packages.

Thank you!

Thank you, thank you, thank you!! A newbie to python, I wasted half a day trying to install twill. 5 minutes with this tutorial, and I was up and running!

YAY

Hey davina, I am glad this helped you. good luck in your python endeavors.

Thanks

Thanks for your help. I could install the plugin for Gimp Palette Generator. Greetings memhet

Thank you so much

I have been trying to work with paramiko for a long time. My web server is linux and my working platform is windows. I need to get some files from windows machine to the server using python and had this installation problem all the time. After trying several things I finally left that idea and started using winscp software. After several months, I found your article and it is so helpful. I just installed paramiko and about to use this. Thank you.

easy_install not working

I followed the steps for easy_install and get as far as getting it to work without installing a package but when I try to install "nose" I get: "ImportError: cannot import name compose_version_string" I get this if I try to install other packages as well. any advice?! James

Hey James, I have never come

Hey James, I have never come across this error on 2.5.4 or 2.6. it seems that compose_version_string comes from pkg_resources.py in site-packages according to all knowing google. I'd be willing to help you, but I would need to see more information about the error and your system.

Same Error

I'm getting the same error as James when I try to install the bbfreeze package, I am also getting a warning when I run easy_install

C:\>easy_install

C:\Python25\Scripts\easy_install-script.py:5: UserWarning: Module site was already imported from C:\Python25\lib\site.pyc, but c:\python25\lib\site-packages\enstaller-3.1.0-py2.5.egg is being added to sys.path

from pkg_resources import load_entry_point

C:\Python25\Scripts\easy_install-script.py:5: UserWarning: Module pkg_resources was already imported from C:\Python25\lib\site-packages\pkg_resources.pyc, but c:\python25\lib\site-packages\enstaller-3.1.0-py2.5.egg is being added to sys.path

hopefully this is enough info that someone will be able to help.

Well...

I'm not familiar with bbfreeze but i tried to reproduce the error but could not(installed without issue), however I noticed in your trace a reference to Enstaller. So I tried to install that and I got an access denied when it tried to replace the easy_install.exe as part of it's install. This makes sense because the file it's trying to replace is actually running. I don't know about this project either. But it looks to be worth checking out, but it makes me wonder what exactly it adds as an enhancement and whether it is different enough to cause these kinds of issues you are seeing. After trying to install enstaller I am seeing the warnings that you indicated above. But no error about version_string

Reinstalled, problem fixed

I tried reinstalling python and my other packages and that fixed the problem. Still not sure what caused it in the first place, but oh well.

why python 2.5.4 ?

Why that version ?We're in 2009 and Python3000 is on.

Soon as my favorite libs are

Soon as my favorite libs are moved over and my favorite linux distro puts it in there as default , I'll move over. Guido's Keynote at this years pycon describes the situation pretty accurately. http://pycon.blip.tv/file/1947431/

distutils.cfg, not disutils.cfg

The configuration file to tell easy_install to use mingw32 compiler is distutils.cfg, not disutils.cfg

oops...good catch

Sorry about that, I made the correction. Thanks

How about using free VS Express instead of VS 2003

I already had Visual Studio Express edition in the path, but easy_install failed to recognize it, so had to install mingw, and the instructions were very helpful, so, thank you. I am wondering if anyone was able to get easy_install to work with the express edition at all.

Glad this helped. I normally

Glad this helped. I normally have vs.net installed in some version on my windows VM, and I could never get easy_install to pick that up.

The way I figure it, the path of least resistance is mingw, however, I'm sure whatever c extensions you end up depending on for a project could probably be optimized better through other compiling means. This would be something to worry about when it's time to cut a release, and I would probably grab the binaries from the author and distribute along with the app, if I had permission of course.

Thank you!

I've just been bashing through installing pycrypto, doing exactly the above one step at a time. Just a shame I didn't find it three hours ago.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd><p> <h1><h2><h3><h4><h5><h6> <img>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>. Beside the tag style "<foo>" it is also possible to use "[foo]".
  • E-Mail addresses are hidden with reCAPTCHA Mailhide.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.