This post is mostly used to document my current Python setup before I get my next Mac where I am going to venture into new setup using pipenv. So far I have had a setup strategy using virtualenvs and virtualenvs only but I have had tests with pipenv and it seems to solve some of the issues I’ve had with my setup.

For background, Python has deservedly a bad reputation about dependencies management. Sooner or later you will get things tangled up and in tears. R people mock us for many reasons but there is no excuses here: Python external modules management is a mess and none of the solutions are very good. This means every Pythonista finds their own set up that kind of does what is needed but still is lacking. Following is my set up that I’ve ended up with, not because it is perfect but because it is where I am at now.

Principles

My setup is built on following principles or guidelines:

• Dont mess up with system Python. When you type in python you will get trusty old Python 2.7 only invited to system management tasks and nothing else
• Don’t install anything outside virtualenvironments. No Anaconda. No Homebrew Python. No binaries. No modules installed to system Python. Just say no
• Virtualenv and pip are only things that install anything (exception: you may of course run setup.py but even that only inside a virtualenv)

These principles make it easier to go back to a working set up when something goes awry. And something will.

Following these principles it means that in the beginning you will have just system Python, pip Installation — pip 20.2.4 documentation and virtualenv Installation — virtualenv 20.1.1.dev5+g9708b54 documentation

Setting up a Project

When starting a project I will following steps:

1. Create new virtualenv. Install it with a new Python (nowadays 3.8 or 3.9)
2. Activate virtualenv
3. Install dependencies
4. Freeze (pip freeze > requirements.txt)
5. Create an alias or project local script/Makefile target that activates the virtualenv the next time you come back
6. Start writing the code

Additionally, when project is larger I create two virtualenvs: projectname and projectname-dev. The latter will have more stuff for development and testing (for instance coverage and pytest). For quick, local projects all dependencies are in one project.

Prototyping

Additionally, for prototyping I have created a virtualenv called “basics” and a system wide alias that activates it and moves to a directory called “notebooks” and then runs “jupyter notebook” in that directory. This virtualenv has needed tools to quickly try out things in a jupyter notebook from requests to sklearn and numpy.