FWIW.CO.ZA

Blog of Christiaan Rademan.

How To Package Your Python Project

I've recently started working on new open-source projects and decided to publish a post a basic tutorial on how to package your Python code.

This tutorial doesn't describe the only way of doing things, merely one specific approach that I use.

Package/Module Names

Python module/package names should generally follow the following constraints:

  • All lowercase
  • Unique on pypi, even if you don’t want to make your package publicly available (you might want to specify it privately as a dependency later)
  • Underscore-separated or no word separators at all (don’t use hyphens)

File Structure

The structure below

Example:

myproject/
    myproject/
        __init__.py
        a_module.py
    setup.py
    setup.cfg
    docs/
    tests/
    Authors
    README.rst
    LICENSE
    ChangeLog
    MANIFEST.in
    requirements.txt
  • myproject/myproject contains your package with modules.

requirements.txt

"Requirements files" are files containing a list of items to be installed using pip install like so:

$ pip install -r requirements.txt

In our case we use the requirements.txt file in the setup.py with setuptools to install dependencies.

MANIFEST.in

The myproject/MANIFEST.in includes additional files to the package.

include Authors
include README.rst
include LICENSE
include requirements.txt

setup.py

import os

try:
    from setuptools import setup
    from setuptools import find_packages
except Exception as e:
    print("Requires 'setuptools'")
    print(" pip install setuptools")
    exit()

config = {
    "name": "myproject",
    "version": "0.0.0",
    "author": "Christiaan F Rademan",
    "author_email": "chris@fwiw.co.za",
    "description": "MyProject Package Example",
    "license": "BSD 3-Clause",
    "keywords": "another example",
    "url": "http://www.fwiw.co.za",
    "packages": find_packages(),
    "include_package_data": True,
    "classifiers": [
        "Topic :: Software Development :: Libraries :: Application Frameworks",
        "Environment :: Other Environment",
        "Intended Audience :: Information Technology",
        "Intended Audience :: System Administrators",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: BSD License",
        "Operating System :: POSIX :: Linux",
        "Programming Language :: Python",
        "Programming Language :: Python :: 2.7"
        ]
    }

# allow setup.py to be run from any path
os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))

if os.path.exists(os.path.join(os.path.dirname(__file__),
                               'requirements.txt')):
    with open(os.path.join(os.path.dirname(__file__),
                           'requirements.txt')) as x:
        requirements = x.read().splitlines()
else:
    requirements = []

with open(os.path.join(os.path.dirname(__file__), 'README.rst')) as x:
    readme = x.read()

print("%s %s\n" % (config['name']))

setup(
    install_requires=requirements,
    long_description=readme,
    **config
)

You can use setup.py to register your project on PyPI which is explained later or use it to remove, install your package.

$ python setup.py install

will install the package

$ python setup.py develop

The develop will not install the package but it will create a .egg-link in the deployment directory back to the project source code directory.

So it's like installing but instead of copying to the site-packages it adds a symbolic link (the .egg-link acts as a multiplatform symbolic link).

That way you can edit the source code and see the changes directly without having to reinstall every time that you make a little change. This is useful when you are the developer of that project hence the name develop.

setup.cfg

This tells PyPI where your README file is.

[metadata]
description-file = README.rst

LICENSE.txt

This file will contain whichver license you want your code to have. I tend to use the BSD 3-Clause license.

What is PyPI?

From the official website:

PyPI — the Python Package Index

The Python Package Index is a repository of software for the Python programming language.

Upload your code on PyPI. It's a big list of python packages that you absolutely must submit your package to for it to be easily one-line installable.

PyPI Account

On PyPI Live and also on PyPI Test, you must create an account in order to be able to upload your code. I recommend using the same email/password for both accounts, just to make your life easier when it comes time to push.

~.pypirc configuration file

[distutils]
index-servers =
  pypi
  pypitest

[pypi]
repository=https://pypi.python.org/pypi
username=your_username

[pypitest]
repository=https://testpypi.python.org/pypi
username=your_username

Upload your package to PyPI Test

This will attempt to register your package against PyPI's test server, just to make sure you've set up everything correctly.

$ python setup.py register -r pypitest

Now upload your package

$ python setup.py sdist upload -r pypitest

Upload your package to PyPI Test

Simply run above commands again and replace pypitest with pypi

Error while Uploading

If you receive an error TypeError: cannot concatenate 'str' and 'NoneType' objects this is a known bug. Its because the upload is not prompting for the password.

To work around this register and upload at the same time like so:

$ python setup.py sdist register -r pypi upload -r pypi