If you have some useful Python modules that you think others might benefit from, but aren’t sure how to go about packaging them up and distributing them, then this short document is for you. By the end of it, you’ll be a contributor to the The Python Package Index (PyPI).
For a more detailed look at packaging a larger project, see this example.
Let’s begin.
Suppose you’ve written a couple modules to help you keep track of your towel (location.py and utils.py), and you’d like to share them. First thing to do is come up with a CamelCase project name for them. Let’s go with “TowelStuff” since it seems appropriate and also it has not yet been used on the The Python Package Index (PyPI).
“TowelStuff” will be the name of our project as well as the name of our distribution. We should also come up with a package name within which our modules will reside (to avoid naming conflicts with other modules). For this example, there’s only one package, so let’s reuse the project name and go with “towelstuff”. Make the layout of your project directory (described below) look like this:
TowelStuff/
bin/
CHANGES.txt
docs/
LICENSE.txt
MANIFEST.in
README.txt
setup.py
towelstuff/
__init__.py
location.py
utils.py
test/
__init__.py
test_location.py
test_utils.py
Here’s what you should do for each of those listed above:
For now, the CHANGES.txt file should only contain:
v<version>, <date> -- Initial release.
since this is your very first version (version number will be described below) and there are no changes to report.
The MANIFEST.in file should contain this:
include *.txt
recursive-include docs *.txt
The README.txt file should be written in reST so that the PyPI can use it to generate your project’s PyPI page. Here’s a 10-second intro to reST that you might use to start with:
===========
Towel Stuff
===========
Towel Stuff provides such and such and so and so. You might find
it most useful for tasks involving <x> and also <y>. Typical usage
often looks like this::
#!/usr/bin/env python
from towelstuff import location
from towelstuff import utils
if utils.has_towel():
print "Your towel is located:", location.where_is_my_towel()
(Note the double-colon and 4-space indent formatting above.)
Paragraphs are separated by blank lines. *Italics*, **bold**,
and ``monospace`` look like this.
A Section
=========
Lists look like this:
* First
* Second. Can be multiple lines
but must be indented properly.
A Sub-Section
-------------
Numbered lists look like you'd expect:
1. hi there
2. must be going
Urls are http://like.this and links can be
written `like this <http://www.example.com/foo/bar>`_.
You might also consider adding a “Contributors” section and/or a “Thanks also to” section to list the names of people who’ve helped.
By the way, to see how the above README.txt looks rendered in html, see the TowelStuff project at the PyPI.
setup.py – Create this file and make it look like this:
from distutils.core import setup
setup(
name='TowelStuff',
version='0.1.0',
author='J. Random Hacker',
author_email='jrh@example.com',
packages=['towelstuff', 'towelstuff.test'],
scripts=['bin/stowe-towels.py','bin/wash-towels.py'],
url='http://pypi.python.org/pypi/TowelStuff/',
license='LICENSE.txt',
description='Useful towel-related stuff.',
long_description=open('README.txt').read(),
install_requires=[
"Django >= 1.1.1",
"caldav == 0.1.4",
],
)
but, of course, replace the towel stuff with your own project and package names. For more details about picking version numbers, see versioning, but ‘0.1.0’ will work just fine for a first release (this is using the common “major.minor.micro” numbering convention).
Use the install_requires argument to automatically install dependencies when your package will be installed and include information about dependencies (so that package management tools like Pip can use the information). It takes a string or list of strings containing requirement specifiers.
The syntax consists of a project’s PyPI name, optionally followed by a comma-separated list of version specifiers. Modern packaging tools implement version specifiers syntax described in PEP 345 and resolve version comparison in compliance with PEP 386.
If you have no scripts to distribute (and thus no bin dir), you can remove the above line which begins with “scripts”.
For our example, TowelStuff does not depend upon any other distributions (it only depends upon what’s already in the Python standard library). To specify dependencies upon other distributions, see the more detailed Project example.
Create your distribution file like so:
$ cd path/to/TowelStuff
$ python setup.py sdist
Running that last command will create a MANIFEST file in your project directory, and also a dist and build directory. Inside that dist directory is the distribution that you’ll be uploading to the PyPI. In our case, the distribution file will be named TowelStuff-0.1.0.tgz. Feel free to poke around in the dist directory to look at your distribution.
Before uploading you first need to create an account at http://pypi.python.org/pypi . Once that’s complete, register your distribution at the PyPI like so:
$ cd path/to/TowelStuff
$ python setup.py register
Use your existing login (choice #1). It will prompt you to save the login info for future use (to which I agree). Then upload:
$ python setup.py sdist upload
This builds the distribution one last time and then uploads it.
Thanks for your contribution!
Down the road, after you’ve made updates to your distribution and wish to make a new release:
Entry points are a Setuptools/Distribute feature that’s really handy in one specific case: register something under a specific key in package A that package B can query for.
Distribute itself uses it. If you’re packaging your project up properly, you’ve probably used the console_scripts entry point:
setup(name='zest.releaser',
...
entry_points={
'console_scripts':
['release = zest.releaser.release:main',
'prerelease = zest.releaser.prerelease:main',
]}
)
console_scripts is an entry point that Setuptools looks up. It looks up all entry points registered under the name console_scripts and uses that information to generate scripts. In the above example that’d be a bin/release script that runs the main() method in zest/releaser/release.py.
You can use that for your own extension mechanism. For zest.releaser I needed some extension mechanism. I wanted to be able to do extra things on prerelease/release/postrelease time.
An entry point for zest.releaser is configured like this in your setup.py:
entry_points={
'console_scripts':
['myscript = my.package.scripts:main'],
'zest.releaser.prereleaser.middle':
['dosomething = my.package.some:some_entrypoint, ]
}
Replace prereleaser and middle in zest.releaser.prereleaser.middle with prerelease/release/postrelease and before/middle/after where needed. (For this specific zest.releaser example).
Now, how to use this in your program? The best way is to show a quick example from zest.releaser where we query and use one of our entry points:
import pkg_resources
...
def run_entry_point(data):
# Note: data is zest.releaser specific: we want to pass
# something to the plugin group = 'zest.releaser.prerelease.middle'
for entrypoint in pkg_resources.iter_entry_points(group=group):
# Grab the function that is the actual plugin.
plugin = entrypoint.load() # Call the plugin
plugin(data)
So: pretty easy and simple way to allow other packages to register something that you want to know. Extra plugins, extra render methods, extra functionality you want to register in your web application, etcetera.