From script to project#
Typical steps#
Organize your script into a package and modules
Add install scripts/instructions
Add documentation and make it available online
Add tests
Deploy your application/library
The final version of this lecture are available here:
Source code: UiO-IN3110/monty_hall_game
Online documentation: https://uio-monty-hall-pre.readthedocs.io
Automatic testing: UiO-IN3110/monty_hall_game
Our test case#
We will use the Monty Hall Game implementation from last week as an example.
This project currently consists of the game file itself and some html templates:
monty_hall_game/
game_server.py # Start web server
templates/*.html # Templates for web-server
Goal: Make the game available as a project with:
a command line interface
a web interface
online and offline documentation
tests
error handling.
Step 1. Organize your script into modules and functions.#
We would like to offer a command line interface and a web interface. To achieve this, we separate the game logic into a separate package (which is simply a directory with Python modules).
myproject/
monty_hall_game/ # Game package
__init__.py # Init file module
monty_hall_game.py # Core game module
The core game module contains the class MontyHallGame, which implements the core functionality of the game. Here is a the user interface for the game package:
Let’s look at the user interface of the package.
Setting up the game:
!tree monty-hall-game0
monty-hall-game0
├── game_server.py
└── templates
├── final.html
├── reselect1.html
├── reselect.html
├── select2.html
└── select.html
1 directory, 6 files
!tree -I __pycache__ monty-hall-game1
monty-hall-game1
├── bin
│ ├── play_monty_hall_cli.py
│ └── play_monty_hall_web.py
├── monty_hall_game
│ ├── __init__.py
│ └── monty_hall_game.py
└── templates
├── final.html
├── reselect1.html
├── reselect.html
├── select2.html
└── select.html
3 directories, 9 files
import os
import sys
sys.path.insert(0, os.path.join(os.getcwd(), "monty-hall-game1"))
import monty_hall_game
game = monty_hall_game.MontyHallGame()
We can now play one round:
game.select_door(door=1)
game.let_host_open_door()
3
game.select_door(1)
game.open_door() # True == win, False == lose
False
Printing the game statistics:
print(game.statistics())
Changed and won: 0 out of 0
Not changed and won: 0 out of 1
With this package, we can build scripts that expose the game to the user via the command line and the web-interface.
We implement these in the folder bin (for binary files. We use this name of convention reasons, even though our files are not really bindary files).
Our new directory structure is:
myproject/
bin/ # Scripts
play_monty_hall_cli.py # Start game with command line interface
play_monty_hall_web.py # Start game with web-server
monty_hall_game/ # Game package
__init__.py # Init file module
monty_hall_game.py # Core game module
templates/*.html # Templates for web-server
Let’s look at the code in more detail (see files in monty-hall-game1 folder)
Step 2. Add installation files and instructions#
Record dependencies with requirements.txt#
Our project has some dependencies to run it, such as flask,
and other dependencies to “develop” it such as pytest and (later) sphinx:
!cat monty-hall-game4/requirements.txt
flask
!cat monty-hall-game4/dev-requirements.txt
pytest
sphinx
The dependencies can be automatically installed with
%pip install -r monty-hall-game4/requirements.txt -r monty-hall-game4/dev-requirements.txt
Requirement already satisfied: flask in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from -r monty-hall-game4/requirements.txt (line 1)) (3.0.0)
Requirement already satisfied: pytest in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from -r monty-hall-game4/dev-requirements.txt (line 1)) (7.4.3)
Requirement already satisfied: sphinx in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from -r monty-hall-game4/dev-requirements.txt (line 2)) (7.2.6)
Requirement already satisfied: click>=8.1.3 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->-r monty-hall-game4/requirements.txt (line 1)) (8.1.7)
Requirement already satisfied: Werkzeug>=3.0.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->-r monty-hall-game4/requirements.txt (line 1)) (3.0.1)
Requirement already satisfied: itsdangerous>=2.1.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->-r monty-hall-game4/requirements.txt (line 1)) (2.1.2)
Requirement already satisfied: blinker>=1.6.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->-r monty-hall-game4/requirements.txt (line 1)) (1.7.0)
Requirement already satisfied: Jinja2>=3.1.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->-r monty-hall-game4/requirements.txt (line 1)) (3.1.2)
Requirement already satisfied: packaging in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from pytest->-r monty-hall-game4/dev-requirements.txt (line 1)) (23.2)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from pytest->-r monty-hall-game4/dev-requirements.txt (line 1)) (1.1.3)
Requirement already satisfied: iniconfig in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from pytest->-r monty-hall-game4/dev-requirements.txt (line 1)) (2.0.0)
Requirement already satisfied: tomli>=1.0.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from pytest->-r monty-hall-game4/dev-requirements.txt (line 1)) (2.0.1)
Requirement already satisfied: pluggy<2.0,>=0.12 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from pytest->-r monty-hall-game4/dev-requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: alabaster<0.8,>=0.7 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (0.7.13)
Requirement already satisfied: sphinxcontrib-htmlhelp>=2.0.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.0.4)
Requirement already satisfied: docutils<0.21,>=0.18.1 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (0.20.1)
Requirement already satisfied: requests>=2.25.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.31.0)
Requirement already satisfied: babel>=2.9 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.13.1)
Requirement already satisfied: sphinxcontrib-applehelp in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.0.7)
Requirement already satisfied: sphinxcontrib-jsmath in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: sphinxcontrib-qthelp in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.0.6)
Requirement already satisfied: sphinxcontrib-devhelp in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.0.5)
Requirement already satisfied: Pygments>=2.14 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.16.1)
Requirement already satisfied: sphinxcontrib-serializinghtml>=1.1.9 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.1.9)
Requirement already satisfied: imagesize>=1.3 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (1.4.1)
Requirement already satisfied: snowballstemmer>=2.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.2.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from Jinja2>=3.1.2->flask->-r monty-hall-game4/requirements.txt (line 1)) (2.1.3)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from requests>=2.25.0->sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2.1.0)
Requirement already satisfied: idna<4,>=2.5 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from requests>=2.25.0->sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from requests>=2.25.0->sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (3.3.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from requests>=2.25.0->sphinx->-r monty-hall-game4/dev-requirements.txt (line 2)) (2023.7.22)
Note: you may need to restart the kernel to use updated packages.
Setuptools#
Further, we can create a setup file to simplify the installation of our game.
First thing we need is a pyproject.toml file:
%pycat monty-hall-game4/pyproject.toml
[build-system]
requires = [
"setuptools",
]
build-backend = "setuptools.build_meta"
[project]
version = "0.1.0"
requires-python = ">=3.8"
license = {text = "MIT License"}
name = "monty_hall_game"
description = "..."
readme = "README.md"
dynamic = ["dependencies", "optional-dependencies"]
[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt",]}
optional-dependencies.dev = { file = ["dev-requirements.txt"] }
[project.scripts]
monty-hall-cli = "monty_hall_game.cli:main"
monty-hall-web = "monty_hall_game.web:main"
We note that we here have used a few new features compared to what we covered in Structuring Python code.
dynamic: Some fields can be modified by other packages, scripts etc. when installing. One example is the list of dependencies, that we now letsetuptoolsfetch fromrequirements.txt. See Dynamic metadata for more information on this. Similarly, we do the same thing for the[dev]extensions that are optional dependencies, only used to build the web client, but not run themonty_hall_game.[project-scripts]: creates an alias from calling themonty_hall_game.cli.main()function from the command line asmonty-hall-cli. The syntax ispackage.submodule:function.
We can now install the monty_hall_game package with
python3 -m pip install .
Using the Python package manager pip has the advantage that we can uninstall the package again:
python3 -m pip uninstall monty_hall_game
!tree monty-hall-game4/
monty-hall-game4/
├── build
│ ├── bdist.linux-x86_64
│ └── lib
│ └── monty_hall_game
│ ├── cli.py
│ ├── game_exceptions.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── monty_hall_game.py
│ └── web.py
├── dev-requirements.txt
├── docs
│ ├── api
│ │ ├── index.rst
│ │ └── monty_hall_game.rst
│ ├── _build
│ │ ├── doctrees
│ │ │ ├── api
│ │ │ │ ├── index.doctree
│ │ │ │ └── monty_hall_game.doctree
│ │ │ ├── environment.pickle
│ │ │ └── index.doctree
│ │ └── html
│ │ ├── api
│ │ │ ├── index.html
│ │ │ └── monty_hall_game.html
│ │ ├── genindex.html
│ │ ├── index.html
│ │ ├── _modules
│ │ │ ├── index.html
│ │ │ └── monty_hall_game
│ │ │ ├── game_exceptions.html
│ │ │ └── monty_hall_game.html
│ │ ├── objects.inv
│ │ ├── py-modindex.html
│ │ ├── search.html
│ │ ├── searchindex.js
│ │ ├── _sources
│ │ │ ├── api
│ │ │ │ ├── index.rst.txt
│ │ │ │ └── monty_hall_game.rst.txt
│ │ │ └── index.rst.txt
│ │ └── _static
│ │ ├── alabaster.css
│ │ ├── basic.css
│ │ ├── custom.css
│ │ ├── doctools.js
│ │ ├── documentation_options.js
│ │ ├── file.png
│ │ ├── language_data.js
│ │ ├── minus.png
│ │ ├── plus.png
│ │ ├── pygments.css
│ │ ├── searchtools.js
│ │ └── sphinx_highlight.js
│ ├── conf.py
│ ├── index.rst
│ ├── make.bat
│ └── Makefile
├── monty_hall_game
│ ├── cli.py
│ ├── game_exceptions.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── monty_hall_game.py
│ ├── __pycache__
│ │ ├── game_exceptions.cpython-310.pyc
│ │ ├── __init__.cpython-310.pyc
│ │ └── monty_hall_game.cpython-310.pyc
│ ├── static
│ │ ├── car.jpg
│ │ ├── door1.png
│ │ ├── door2.png
│ │ ├── door3.png
│ │ ├── goat.jpg
│ │ ├── goat_lost.jpg
│ │ └── winner.png
│ ├── templates
│ │ ├── final.html
│ │ ├── layout7.html
│ │ ├── reselect.html
│ │ └── select.html
│ └── web.py
├── monty_hall_game.egg-info
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── PKG-INFO
│ ├── requires.txt
│ ├── SOURCES.txt
│ └── top_level.txt
├── pyproject.toml
├── README.md
├── readthedocs.yml
├── requirements.txt
├── runtime.txt
└── tests
├── __pycache__
│ └── test_game_module.cpython-310-pytest-7.4.3.pyc
└── test_game_module.py
23 directories, 76 files
Installation instructions#
Finally it is good practice to add a installation instructions to the README.md file:
from IPython.display import Markdown, display
with open("monty-hall-game4/README.md") as f:
display(Markdown(f.read()))
Monty Hall Game
This repository contains a simple implementation of the Monty Hall Game with a command line and web interface.
Installation
Install the game with
python3 -m pip install .
Running the game
The command line interface is started with:
monty-hall-cli
or with
python3 -m monty_hall_game
The web server is started with:
monty-hall-web
Documentation
The documentation is available online, or can be built locally with
cd docs
make html
firefox _build/html/index.html
New files#
monty_hall_game/
README.md # Installation instructions
requirements.txt # List of project dependencies
dev-requirements.txt # List of development dependencies
pyproject.toml # SetupTools file
!pwd
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project
%cd monty-hall-game4/
%pip install .
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4
Processing /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4
Installing build dependencies ... ?25ldone
?25h Getting requirements to build wheel ... ?25ldone
?25h Installing backend dependencies ... ?25ldone
?25h Preparing metadata (pyproject.toml) ... ?25ldone
?25hRequirement already satisfied: flask in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from monty-hall-game==0.1.0) (3.0.0)
Requirement already satisfied: itsdangerous>=2.1.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->monty-hall-game==0.1.0) (2.1.2)
Requirement already satisfied: Jinja2>=3.1.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->monty-hall-game==0.1.0) (3.1.2)
Requirement already satisfied: Werkzeug>=3.0.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->monty-hall-game==0.1.0) (3.0.1)
Requirement already satisfied: blinker>=1.6.2 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->monty-hall-game==0.1.0) (1.7.0)
Requirement already satisfied: click>=8.1.3 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from flask->monty-hall-game==0.1.0) (8.1.7)
Requirement already satisfied: MarkupSafe>=2.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from Jinja2>=3.1.2->flask->monty-hall-game==0.1.0) (2.1.3)
Building wheels for collected packages: monty-hall-game
Building wheel for monty-hall-game (pyproject.toml) ... ?25ldone
?25h Created wheel for monty-hall-game: filename=monty_hall_game-0.1.0-py3-none-any.whl size=5515 sha256=267eda1931ab96387337dfbd972aecb677e1545af65dc49bffd5fd899fb2b463
Stored in directory: /home/dokken/.cache/pip/wheels/09/58/ce/9a5cb9c9e406dd150c303a550c32272fea1f7295aefd30c59a
Successfully built monty-hall-game
Installing collected packages: monty-hall-game
Attempting uninstall: monty-hall-game
Found existing installation: monty-hall-game 0.1.0
Uninstalling monty-hall-game-0.1.0:
Successfully uninstalled monty-hall-game-0.1.0
Successfully installed monty-hall-game-0.1.0
Note: you may need to restart the kernel to use updated packages.
Step 3. Add documentation#
We should add docstrings to the module monty_hall_game.py file.
Note, that I write the documentation in a Sphinx Markup Style (see https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html) to obtain nicely rendered online documentation.
Let’s look at the code in more detail (see files in monty-hall-game2 folder)
Once done, we can access the docstrings as usual:
# update import path and re-import monty_hall_game
sys.path.insert(0, os.path.join(os.getcwd(), "monty-hall-game2"))
from importlib import reload
reload(monty_hall_game)
reload(monty_hall_game.monty_hall_game)
reload(monty_hall_game)
from monty_hall_game.monty_hall_game import MontyHallGame
MontyHallGame.let_host_open_door?
Signature: MontyHallGame.let_host_open_door(self)
Docstring: <no docstring>
File: ~/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game1/monty_hall_game/monty_hall_game.py
Type: function
Python documentation with Sphinx#
Sphinx is a powerful tool to create documentation for Python projects and provides more flexibiliy.
Installation#
python3 -m pip install sphinx
How to get started#
Use the quick start command to configure a base Sphinx documentation
sphinx-quickstart
Among other things, the quickstart guide will ask for the documentation folder. I typically choose
docsfor this.Use
mkdir -p docs/api sphinx-apidoc -o docs/api monty_hall_game
to add documentation for each module.
Edit
docs/index.rstto change the content of your main page.Compile the documentation with:
cd docs make html
(make sure that the module is in sys.path or installed).
The documentation is available on
docs/_build/html/index.html.
%cd docs
!make html
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4/docs
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v7.2.6
WARNING: Invalid configuration value found: 'language = None'. Update your configuration to a valid language code. Falling back to 'en' (English).
WARNING: html_static_path entry '_static' does not exist
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
writing output...
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] api/monty_hall_game
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets... copying static files... done
copying extra files... done
done
writing output... [100%] indexame
generating indices... genindex py-modindex done
highlighting module code... [100%] monty_hall_game.monty_hall_game
writing additional pages... search done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 2 warnings.
The HTML pages are in _build/html.
Build finished. The HTML pages are in _build/html.
New files (autogenerated with sphinx-quickstart and sphinx-apidoc)#
docs/
conf.py # Sphinx configuration file
index.rst # Index page (in markdown format)
make.bat # Windows build file
Makefile # Linux/MacOS build file
docs/api
modules.rst # Module page
monty_hall_game.rst # Module page
Step 4. Add tests#
We will use pytest as testing framework.
New files:
monty_hall_game/
tests
test_game_module.py
We can run the test suite with
Let’s look at the code in more detail (see files in monty-hall-game4 folder)
%cd ..
!python3 -m pytest tests
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4
collected 3 items
tests/test_game_module.py ... [100%]
============================== 3 passed in 0.01s ===============================
Continuous integration with GitHub Actions#
Quick guide to GitHub action#
create .github/workflows/test.yml (any name.yml will do, you can have several) with steps:
checkout the repo
install Python
install your package and its dependencies
run the tests
# This is a GitHub workflow defining a set of jobs with a set of steps.
# ref: https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions
#
name: Tests
on:
pull_request:
push:
jobs:
test:
runs-on: ubuntu-22.04
strategy:
matrix:
python:
- "3.9"
- "3.10"
steps:
# checkout the repository
- uses: actions/checkout@v3
# setup Python
- name: Install Python ${{ matrix.python }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python }}
# preserve pip cache to speed up installation
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
# Look to see if there is a cache hit for the corresponding requirements file
key: ${{ runner.os }}-pip-${{ hashFiles('*requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install --upgrade .[dev]
pip freeze
- name: Run tests
run: |
pytest -v --color=yes
Publishing the documentation#
Similarly, we can use Github actions to publish a webpage hosted by Github. We will first make a stand-alone action that simply uploads the built web-page as an artifact. This can be downloaded an inspected.
ame: Build documentation
on:
workflow_dispatch:
workflow_call:
pull_request:
branches:
- main
jobs:
build_docs:
runs-on: ubuntu-22.04
env:
PYTHON_VERSION: "3.10"
steps:
# checkout the repository
- uses: actions/checkout@v4
# setup Python
- name: Install Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
# preserve pip cache to speed up installation
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
# Look to see if there is a cache hit for the corresponding requirements file
key: ${{ runner.os }}-pip-${{ hashFiles('*requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade .[dev]
python -m pip freeze
- name: Build documentation
run: |
cd docs
make html
- name: Upload documentation artifact
uses: actions/upload-artifact@v3
with:
name: documentation
path: ./docs/_build/html
if-no-files-found: error
By adding workflow_call to the settings that decide when this is executed, we allow this workflow to be run from inside another workflow.
We can then create a Github publishing workflow, that only runs on the main branch, that pushes the web page to Github
name: Github Pages
on:
push:
branches: [main]
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true
jobs:
build-docs:
uses: ./.github/workflows/build_docs.yml
deploy:
needs: [build-docs]
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Download docs artifact
# docs artifact is uploaded by build-docs job
uses: actions/download-artifact@v3
with:
name: documentation
path: "./public"
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
with:
path: "./public"
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
Check:
Check the build status page to see if your build passes or fails according to the return status of the build command by visiting your GitHub repository and selecting actions.
Current directory structure#
monyty_hall_game/
README.md # Installation instructions
requirements.txt # Dependencies
setup.py # Setuptools
monty_hall_game/ # Main module
__init__.py
web.py
cli.py
game_exceptions.py
monty_hall_game.py
templates/*.html
static/*.{png,jpg}
docs/ # Sphinx documentation (mostly autogenerated)
conf.py
index.rst
Makefile
modules.rst
monty_hall_game.rst
tests # tests in pytest format
test_game_module.py
.github/workflows/test.yml # continuous integration
.gitignore
Deploy your Python library to PyPI#
Wouldn’t it be nice if your users could just type in:
python3 -m pip install monty_hall_game
This is possible by uploading the package to the Python Package Index (PyPI).
Create a source and wheel distribution with the following command:
python -m pip install build
python -m build .
Upload the distribution to pypi (will ask for the pypi credentials) with:
twine upload -r test dist/*
Result see https://test.pypi.org/project/uio-monty-hall-game/
%cd monty-hall-game4
!rm -rf dist
!python3 -m pip install build
!python3 -m build .
[Errno 2] No such file or directory: 'monty-hall-game4'
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4
/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages/IPython/core/magics/osm.py:393: UserWarning: using bookmarks requires you to install the `pickleshare` library.
bkms = self.shell.db.get('bookmarks', {})
Requirement already satisfied: build in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (1.0.3)
Requirement already satisfied: packaging>=19.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from build) (23.2)
Requirement already satisfied: tomli>=1.1.0 in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from build) (2.0.1)
Requirement already satisfied: pyproject_hooks in /home/dokken/Documents/src/UiO/UiO-IN3110.github.io/monty-hall/lib/python3.10/site-packages (from build) (1.0.0)
* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools)
* Getting build dependencies for sdist...
running egg_info
writing monty_hall_game.egg-info/PKG-INFO
writing dependency_links to monty_hall_game.egg-info/dependency_links.txt
writing entry points to monty_hall_game.egg-info/entry_points.txt
writing requirements to monty_hall_game.egg-info/requires.txt
writing top-level names to monty_hall_game.egg-info/top_level.txt
reading manifest file 'monty_hall_game.egg-info/SOURCES.txt'
writing manifest file 'monty_hall_game.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing monty_hall_game.egg-info/PKG-INFO
writing dependency_links to monty_hall_game.egg-info/dependency_links.txt
writing entry points to monty_hall_game.egg-info/entry_points.txt
writing requirements to monty_hall_game.egg-info/requires.txt
writing top-level names to monty_hall_game.egg-info/top_level.txt
reading manifest file 'monty_hall_game.egg-info/SOURCES.txt'
writing manifest file 'monty_hall_game.egg-info/SOURCES.txt'
running check
creating monty_hall_game-0.1.0
creating monty_hall_game-0.1.0/monty_hall_game
creating monty_hall_game-0.1.0/monty_hall_game.egg-info
creating monty_hall_game-0.1.0/tests
copying files to monty_hall_game-0.1.0...
copying README.md -> monty_hall_game-0.1.0
copying dev-requirements.txt -> monty_hall_game-0.1.0
copying pyproject.toml -> monty_hall_game-0.1.0
copying requirements.txt -> monty_hall_game-0.1.0
copying monty_hall_game/__init__.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game/__main__.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game/cli.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game/game_exceptions.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game/monty_hall_game.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game/web.py -> monty_hall_game-0.1.0/monty_hall_game
copying monty_hall_game.egg-info/PKG-INFO -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying monty_hall_game.egg-info/SOURCES.txt -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying monty_hall_game.egg-info/dependency_links.txt -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying monty_hall_game.egg-info/entry_points.txt -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying monty_hall_game.egg-info/requires.txt -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying monty_hall_game.egg-info/top_level.txt -> monty_hall_game-0.1.0/monty_hall_game.egg-info
copying tests/test_game_module.py -> monty_hall_game-0.1.0/tests
Writing monty_hall_game-0.1.0/setup.cfg
Creating tar archive
removing 'monty_hall_game-0.1.0' (and everything under it)
* Building wheel from sdist
* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools)
* Getting build dependencies for wheel...
running egg_info
writing monty_hall_game.egg-info/PKG-INFO
writing dependency_links to monty_hall_game.egg-info/dependency_links.txt
writing entry points to monty_hall_game.egg-info/entry_points.txt
writing requirements to monty_hall_game.egg-info/requires.txt
writing top-level names to monty_hall_game.egg-info/top_level.txt
reading manifest file 'monty_hall_game.egg-info/SOURCES.txt'
writing manifest file 'monty_hall_game.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/monty_hall_game
copying monty_hall_game/cli.py -> build/lib/monty_hall_game
copying monty_hall_game/monty_hall_game.py -> build/lib/monty_hall_game
copying monty_hall_game/game_exceptions.py -> build/lib/monty_hall_game
copying monty_hall_game/__main__.py -> build/lib/monty_hall_game
copying monty_hall_game/__init__.py -> build/lib/monty_hall_game
copying monty_hall_game/web.py -> build/lib/monty_hall_game
running egg_info
writing monty_hall_game.egg-info/PKG-INFO
writing dependency_links to monty_hall_game.egg-info/dependency_links.txt
writing entry points to monty_hall_game.egg-info/entry_points.txt
writing requirements to monty_hall_game.egg-info/requires.txt
writing top-level names to monty_hall_game.egg-info/top_level.txt
reading manifest file 'monty_hall_game.egg-info/SOURCES.txt'
writing manifest file 'monty_hall_game.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/cli.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/monty_hall_game.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/game_exceptions.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/__main__.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/__init__.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
copying build/lib/monty_hall_game/web.py -> build/bdist.linux-x86_64/wheel/monty_hall_game
running install_egg_info
Copying monty_hall_game.egg-info to build/bdist.linux-x86_64/wheel/monty_hall_game-0.1.0-py3.10.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/monty_hall_game-0.1.0.dist-info/WHEEL
creating '/home/dokken/Documents/src/UiO/UiO-IN3110.github.io/lectures/production/from-script-to-project/monty-hall-game4/dist/.tmp-ecy7fq2k/monty_hall_game-0.1.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'monty_hall_game/__init__.py'
adding 'monty_hall_game/__main__.py'
adding 'monty_hall_game/cli.py'
adding 'monty_hall_game/game_exceptions.py'
adding 'monty_hall_game/monty_hall_game.py'
adding 'monty_hall_game/web.py'
adding 'monty_hall_game-0.1.0.dist-info/METADATA'
adding 'monty_hall_game-0.1.0.dist-info/WHEEL'
adding 'monty_hall_game-0.1.0.dist-info/entry_points.txt'
adding 'monty_hall_game-0.1.0.dist-info/top_level.txt'
adding 'monty_hall_game-0.1.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built monty_hall_game-0.1.0.tar.gz and monty_hall_game-0.1.0-py3-none-any.whl
ls dist
monty_hall_game-0.1.0-py3-none-any.whl monty_hall_game-0.1.0.tar.gz
!twine upload -r test dist/*
Uploading distributions to https://test.pypi.org/legacy/
Uploading uio_monty_hall_game-2022.11.17-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 kB • 00:00 • 278.3 kB/s
?25hUploading uio-monty-hall-game-2022.11.17.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 kB • 00:00 • ?
?25h
View at:
https://test.pypi.org/project/uio-monty-hall-game/2022.11.17/
Final directory layout#
!tree -I __pycache__ .
.
├── build
│ ├── bdist.linux-x86_64
│ └── lib
│ └── monty_hall_game
│ ├── cli.py
│ ├── game_exceptions.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── monty_hall_game.py
│ └── web.py
├── dev-requirements.txt
├── dist
│ ├── monty_hall_game-0.1.0-py3-none-any.whl
│ └── monty_hall_game-0.1.0.tar.gz
├── docs
│ ├── api
│ │ ├── index.rst
│ │ └── monty_hall_game.rst
│ ├── _build
│ │ ├── doctrees
│ │ │ ├── api
│ │ │ │ ├── index.doctree
│ │ │ │ └── monty_hall_game.doctree
│ │ │ ├── environment.pickle
│ │ │ └── index.doctree
│ │ └── html
│ │ ├── api
│ │ │ ├── index.html
│ │ │ └── monty_hall_game.html
│ │ ├── genindex.html
│ │ ├── index.html
│ │ ├── _modules
│ │ │ ├── index.html
│ │ │ └── monty_hall_game
│ │ │ ├── game_exceptions.html
│ │ │ └── monty_hall_game.html
│ │ ├── objects.inv
│ │ ├── py-modindex.html
│ │ ├── search.html
│ │ ├── searchindex.js
│ │ ├── _sources
│ │ │ ├── api
│ │ │ │ ├── index.rst.txt
│ │ │ │ └── monty_hall_game.rst.txt
│ │ │ └── index.rst.txt
│ │ └── _static
│ │ ├── alabaster.css
│ │ ├── basic.css
│ │ ├── custom.css
│ │ ├── doctools.js
│ │ ├── documentation_options.js
│ │ ├── file.png
│ │ ├── language_data.js
│ │ ├── minus.png
│ │ ├── plus.png
│ │ ├── pygments.css
│ │ ├── searchtools.js
│ │ └── sphinx_highlight.js
│ ├── conf.py
│ ├── index.rst
│ ├── make.bat
│ └── Makefile
├── monty_hall_game
│ ├── cli.py
│ ├── game_exceptions.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── monty_hall_game.py
│ ├── static
│ │ ├── car.jpg
│ │ ├── door1.png
│ │ ├── door2.png
│ │ ├── door3.png
│ │ ├── goat.jpg
│ │ ├── goat_lost.jpg
│ │ └── winner.png
│ ├── templates
│ │ ├── final.html
│ │ ├── layout7.html
│ │ ├── reselect.html
│ │ └── select.html
│ └── web.py
├── monty_hall_game.egg-info
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── PKG-INFO
│ ├── requires.txt
│ ├── SOURCES.txt
│ └── top_level.txt
├── pyproject.toml
├── README.md
├── readthedocs.yml
├── requirements.txt
├── runtime.txt
└── tests
└── test_game_module.py
22 directories, 74 files
monyty_hall_game/
README.md # Installation instructions
requirements.txt # Dependencies
setup.py # Setuptools
monty_hall_game/ # Our package
__init__.py
game_exceptions.py
monty_hall_game.py
bin/ # Scripts
play_monty_hall_cli.py
play_monty_hall_web.py
templates/*.html
docs/ # Sphinx documentation (mostly autogenerated)
conf.py
index.rst
Makefile
modules.rst
monty_hall_game.rst
tests # tests in pytest format
test_game_module.py
.github/workflows/test.yml # continuous integration
.gitignore
Summary of today’s topics#
Installation
requirements.txt
Package + pyproject.toml
Testing
Continuous integration (with GitHub)
Documentation
Sphinx
Publishing on Github
Deployment / Publishing
Deployment of Python packages with PyPI