Python refresher#
Python is a high-level, interpreted programming language and currently one of the most popular programming languages.
The syntax is by design easy to read and resembles pseudocode.
Python is fast to program but not the fastest programming language (although there are ways to write faster Python code).
Books and tutorials#
Here are some good books and tutorials for Python 3:
Installation#
Python can be installed in different ways - and might be already installed on your system.
Your computer most likely uses Python for its own purposes; this is something you should not interfere with.
It is customary to use tools such as docker, conda and virtual environments to keep track of different versions of Python and installed packages.
We recommend anaconda as a quick way to get ~everything you need for scientific Python.
Advantages:
Includes a Python distribution with package manager and many packages.
Freely available for Windows, mac, and Linux
Install anaconda from https://anaconda.com as a local user.
The conda package manager#
conda can install many types of packages, including Python itself. conda can also install Python packages, which add functionality to Python.
Make a conda environment
conda create -n in3110Activate the conda environment
conda activate in3110.Install Python
conda install python
Check where Python is executing from: which python3.
You can use the conda package manager to find and install new packages:
Search for a package:
conda search scipy
Install a package:
conda install scipy
List all installed packages:
conda list
The pip package manager#
pip is Python’s own package manager, and can install only Python packages.
You can use pip to find and install new packages:
Search for a package:
python3 -m pip search scipy
Install a package:
python3 -m pip install scipy
List all installed packages:
python3 -m pip list
Q: What does python3 -m pip .... do?
This calls the Python-module
pipand its__main__.pyfile (see: Python3 Docs)We can look for other command-line arguments with
python3 --helporman python
Q: What does it actually mean to install a package?
Download the source code
Adding source code to path
! python3 --help
usage: python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b : issue warnings about str(bytes_instance), str(bytearray_instance)
and comparing bytes/bytearray with str. (-bb: issue errors)
-B : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : turn on parser debugging output (for experts only, only works on
debug builds); also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also -? or --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O : remove assert and __debug__-dependent statements; add .opt-1 before
.pyc extension; also PYTHONOPTIMIZE=x
-OO : do -O changes and also discard docstrings; add .opt-2 before
.pyc extension
-q : don't print version and copyright messages on interactive startup
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-u : force the stdout and stderr streams to be unbuffered;
this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option. The following options are available:
-X faulthandler: enable faulthandler
-X showrefcount: output the total reference count and number of used
memory blocks when the program finishes or after each statement in the
interactive interpreter. This only works on debug builds
-X tracemalloc: start tracing Python memory allocations using the
tracemalloc module. By default, only the most recent frame is stored in a
traceback of a trace. Use -X tracemalloc=NFRAME to start tracing with a
traceback limit of NFRAME frames
-X importtime: show how long each import takes. It shows module name,
cumulative time (including nested imports) and self time (excluding
nested imports). Note that its output may be broken in multi-threaded
application. Typical usage is python3 -X importtime -c 'import asyncio'
-X dev: enable CPython's "development mode", introducing additional runtime
checks which are too expensive to be enabled by default. Effect of the
developer mode:
* Add default warning filter, as -W default
* Install debug hooks on memory allocators: see the PyMem_SetupDebugHooks()
C function
* Enable the faulthandler module to dump the Python traceback on a crash
* Enable asyncio debug mode
* Set the dev_mode attribute of sys.flags to True
* io.IOBase destructor logs close() exceptions
-X utf8: enable UTF-8 mode for operating system interfaces, overriding the default
locale-aware mode. -X utf8=0 explicitly disables UTF-8 mode (even when it would
otherwise activate automatically)
-X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted at the
given directory instead of to the code tree
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None'
-X int_max_str_digits=number: limit the size of int<->str conversions.
This helps avoid denial of service attacks when parsing untrusted data.
The default is sys.int_info.default_max_str_digits. 0 disables.
--check-hash-based-pycs always|default|never:
control how Python invalidates hash-based .pyc files
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH : ':'-separated list of directories prefixed to the
default module search path. The result is sys.path.
PYTHONHOME : alternate <prefix> directory (or <prefix>:<exec_prefix>).
The default module search path uses <prefix>/lib/pythonX.X.
PYTHONPLATLIBDIR : override sys.platlibdir.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONUTF8: if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
to seed the hashes of str and bytes objects. It can also be set to an
integer in the range [0,4294967295] to get hash values with a
predictable seed.
PYTHONINTMAXSTRDIGITS: limits the maximum digit characters in an int value
when converting from a string and when converting an int back to a str.
A value of 0 disables the limit. Conversions to or from bases 2, 4, 8,
16, and 32 are never limited.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
on Python memory allocators. Use PYTHONMALLOC=debug to install debug
hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display of
locale coercion and locale compatibility warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
debugger. It can be set to the callable of your debugger of choice.
PYTHONDEVMODE: enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.
PYTHONWARNDEFAULTENCODING: enable opt-in EncodingWarning for 'encoding=None'.
! man python3
Example:#
Download course files and install necessary packages:
git clone https://github.com/UiO-IN3110/UiO-IN3110.github.io/
cd UiO-IN3110.github.io
conda create -n in3110
conda activate in3110
conda install python
python3 -m pip install -r requirements.txt
python3 -m bash_kernel.install --sys-prefix
First Python encounter: A scientific hello world program#
File hw.py in UiO-IN3110.github.io/lectures/python:
#!/usr/bin/env python3
from math import sin
import sys
x = float(sys.argv[1])
print(f"Hello world, sin({x}) = {sin(x)}")
Running the script from the command line#
Works an all operating systems:
> python3 hw.py 0.8
Hello world, sin(0.8) = 0.7173560908995228
A Linux/mac alternative is to make the file executable.
This will only work when the the line with #!... (known as the shebang) is included:
> chmod a+x hw.py
> ./hw.py 10
Hello world, sin(10.0) = -0.5440211108893698
Dissection of hw.py (1)#
On Linux/mac: find out what kind of script language (interpreter) to use and expose all environment variables:
#!/usr/bin/env python3
Access library functionality like the function sin (from the math-module) and the list sys.argv
(of command-line arguments):
from math import sin
import sys
Read first command line argument and convert it to a floating point object:
x = float(sys.argv[1])
Note: Python variables are not declared.
Dissection of hw.py (2)#
Print out the result using a format string:
print(f"Hello world, sin({x}) = {sin(x)}")
or with complete control of the formating of floats (similar to the C’s printf syntax):
print(f"Hello world, sin({x:g}) = {sin(x):.3f}")
Essential Python syntax#
Python as a calculator#
You can use the Python as a simple calculator:
1 + 2
3
4.5 / 3 + (1 + 2) * 3
10.5
Use ** to compute the power:
4**5
1024
Python also supports complex numbers:
a = 1 + 2j
b = 3 - 5j
a * b
(13+1j)
More advanced mathematical functions can be imported:
from math import log10
log10(5)
0.6989700043360189
Python variables and data types#
Basic types#
strings:
"strings for storing text"numbers:
1, 1.5tuples:
(1, 2, 3)for storing static collectionslists:
["a", "b", "c"]for mutable, ordered sequencesdicts:
{"key": "value"}for storing key-value pairssets:
{"do", "re", "mi"}for storing unique, unordered collections
Strings#
Strings can be expressed as single quotes ('...') or double quotes ("...") with the same result:
'some string'
is equivalent to
"some string"
Triple-quoted strings can be multi line with embedded newlines:
text = """large portions of a text
can be conveniently placed inside
triple-quoted strings (newlines
are preserved)"""
Special characters in strings#
Use the backslash \ to escape special characters:
s = '"This is a quote" and \n here comes a backslash: \\'
print(s)
"This is a quote" and
here comes a backslash: \
String concatenation#
Strings can be glued together with the + and the * operators:
"hello " * 3 + "world"
'hello hello hello world'
This works also with string variables:
quote = "I will not eat chips all day"
(quote + ", ") * 10 + quote
'I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day, I will not eat chips all day'
Slicing#
You can extract a sub-string with the [start:end] slicing notation:
quote[2:6]
'will'
If the start (left) argument is left out, the substring will start from the first (last) character:
quote[:6] # I will
quote[7:] # not eat chips all day
'not eat chips all day'
Negative indices can be used to index “from the right”:
+---+---+---+---+---+
| c | h | i | p | s |
+---+---+---+---+---+
0 1 2 3 4
-5 -4 -3 -2 -1
"chips"[1:-2]
'hi'
Python strings cannot be changed#
Python strings are immutable, meaning that they cannot be changed:
quote[1] = "x"
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In [14], line 1
----> 1 quote[1] = "x"
TypeError: 'str' object does not support item assignment
If one wants to change a string, one needs to create a new one:
quote = quote[:1] + "x" + quote[2:]
print(quote)
Ixwill not eat chips all day
More useful string operations#
Code |
Meaning |
|---|---|
‘day’ in quote |
True if string contains substring |
quote.find(‘i’) |
index where first ‘i’ is found |
quote.split() |
split at whitespace (returns list) |
quote.replace(‘chips’, ‘salad’) |
replace all occurances |
quote.lower() |
convert to lower case |
quote.upper() |
convert to upper case |
quote.strip() |
remove leading/trailing blanks |
Note that the modification operations return a new string (since strings are immutable).
Lists#
Python lists allow you to group together a sequence of values:
mylist = ["Hello", "world", "!!!"]
Generally lists are used when you have an ordered collection of the same kind of thing:
filenames
URLs
objects
numbers
Lists do not require items to have the same type, though in practice they usually do
mylist = ["Hello", 4, True]
List operations#
Many of the operations that we know from strings also work on lists, such as indexing:
mylist[0]
'Hello'
slicing:
mylist[1:]
[4, True]
and concatenation:
newlist = mylist + ["!"] * 3
newlist
['Hello', 4, True, '!', '!', '!']
Lists can be changed#
In constrast to strings, lists are mutable and can be changed:
mylist = [11, 12, 14]
mylist[2] = 13
mylist
[11, 12, 13]
We can also append additional items to a list:
mylist.append(14)
mylist
[11, 12, 13, 14]
Cheat sheet for Python lists#
Construction |
Meaning |
|---|---|
a = [] |
initialize an empty list |
a = [1, 4.4, ‘run.py’] |
initialize a list |
a.append(elem) |
add elem object to the end |
a + [1,3] |
add two lists |
a.insert(i, e) |
insert element e before index i |
a[3] |
index a list element |
a[-1] |
get last list element |
a[1:3] |
slice: return sublist (here: index 1, 2) |
del a[3] |
delete an element (index 3) |
a.remove(e) |
remove an element with value e |
a.index(’run.py’) |
find index corresponding to an element’s value |
‘value’ in a |
test if a value is contained in the list |
a.count(v) |
count how many elements have the value v |
len(a) |
number of elements in list a |
min(a) |
the smallest element in a |
max(a) |
the largest element in a |
sum(a) |
add all elements in a |
sorted(a) |
return sorted version of list a |
reversed(a) |
return reversed view version of list a |
b[3][0][2] |
nested list indexing |
isinstance(a, list) |
is True if a is a list |
type(a) is list |
is True if a is a list |
Tuples#
Tuples are very similar to lists, but they are immutable, just like strings.
Functionally, they are essentially immutable lists, but they tend to be used for a different purpose:
a single “thing” with multiple components
Tuples are created with parentheses:
mytuple = ("a string", 2.5, 6, "another string")
Since tuples are immutable we cannot change them:
mytuple[1] = -10 # Error, tuple cannot be changed
Instead we need to create a new tuple with the changed values, for example by converting the tuple to a list, changing it, and converting it back to a tuple:
l = list(mytuple) # convert tuple to list (copy)
l[1:3] = ["is", "not"]
mytuple = tuple(l) # convert list to tuple (copy)
mytuple
('a string', 'is', 'not', 'another string')
Tuple cheat sheet#
Code |
Meaning |
|---|---|
a = () |
initialize an empty tuple |
a = (1, 4.4, ‘run.py’) |
initialize a tuple |
a + (1,3) |
concatenate two tuples (returns a new tuple) |
a[3] |
index a list element |
a[-1] |
get last list element |
a[1:3] |
slice: return subtuple (here: index 1, 2) |
a.index(‘value’) |
find index corresponding to an element’s value |
‘value’ in a |
test if a value is contained in the list |
a.count(v) |
count how many elements have the value v |
len(a) |
number of elements in list a |
min(a) |
the smallest element in a |
max(a) |
the largest element in a |
sum(a) |
add all elements in a |
sorted(a) |
return sorted list with the values of a |
reversed(a) |
return reversed version of a |
b[3][0][2] |
nested list indexing |
isinstance(a, tuple) |
is True if a is a tuple or subclass |
type(a) is tuple |
is True if a is exactly a tuple |
Python dictionaries#
Recall that lists always used integers as indices:
mylist[10]
Python dictionaries are similar but you can use any hashable object as index:
mydict["hallo"] # dictionary can use e.g. a string as indices
Basic dictionary operations#
We create dictionaries with the {} syntax.
For each dictionary entry, we need to provide one (immutable) key and its value:
phonebook = {"John Doe" : 99954329,
"Franz Dahl": 4881221}
mydict = {"1" : "A number",
"house" : "A building to live in",
"kitchen": None}
Once created, we can access the dictionary entries:
phonebook["John Doe"] # 99954329
mydict["tbane"] # gives a KeyError
Dictionaries are mutable, so we can change them:
mydict['somekey'] = 1.0
mydict.update(otherdict) # add/replace key-value pairs
del mydict[2]
del mydict['somekey']
Dictionary cheat sheet#
Construction |
Meaning |
|---|---|
a = {} |
initialize an empty dictionary |
a = {‘point’: (0,0.1), ‘value’: 7} |
initialize a dictionary |
a = dict(point=(2,7), value=3) |
initialize a dictionary w/string keys |
a.update(b) |
add key-value pairs from b in a |
a.update(key1=value1, key2=value2) |
add key-value pairs in a |
a[‘hide’] = True |
add new key-value pair to a |
a[‘point’] |
get value corresponding to key point |
for key in a: |
loop over keys in unknown order |
for key in sorted(a): |
loop over keys in alphabetic order |
‘value’ in a |
True if string value is a key in a |
del a[‘point’] |
delete a key-value pair from a |
list(a.keys()) |
list of keys |
list(a.values()) |
list of values |
len(a) |
number of key-value pairs in a |
isinstance(a, dict) |
is True if a is a dictionary |
Summary: Common data structures#
Numbers:
intfloatcomplex
Sequences:
stringlisttupleset
Mappings:
dict(dictionary/hash)
Control structures in Python#
Conditionals/branching#
if condition:
<block of statements>
elif condition:
<block of statements>
else:
<block of statements>
Also here, condition must be a boolean expression.
Important: Python uses indentation to determine the start/end of blocks (instead of e.g. brackets). In Python, it is common to indent with 4 spaces.
Examples#
Let’s look at an example:
i = 25
if i < 0:
print(f"{i} is a negative number")
elif 0 <= i < 20:
print(f"{i} is a small number")
else:
print(f"{i} is a large number")
25 is a large number
Python variables are strongly typed. We can use if statements to test for a variable’s type:
if isinstance(a, int): # int?
# ...
if isinstance(a, (list, tuple)): # list or tuple?
# ...
while loop#
while condition:
<block of statements>
Here, condition must be a boolean expression (or have a boolean interpretation), for example: i < 10.
for loop#
for element in somelist:
<block of statements>
Here, somelist must be an iterable object, for example a list, tuple, or string.
Example#
Let’s look at an example:
shoppinglist = ["tea", "butter", "milk"]
for item in shoppinglist:
print(f"Remember to buy {item}.")
Remember to buy tea.
Remember to buy butter.
Remember to buy milk.
If you want to iterate over a sequence of numbers, you can use the range command:
for i in range(3):
print(i)
0
1
2
Functions#
Python functions allow you to encapsulate a task - they combine many instructions into a single line of code.
As an example, let’s write a function that splits a string at a given character:
def split(string, char):
"""Split the string at the given character"""
position = string.find(char)
if position > 0:
return string[: position + 1], string[position + 1 :]
else:
return string, ""
So far, we have only defined the function (in cooking this is equivalent of writing down a recipe).
We must call our function to have an actual effect (or equivalently, actually cooking the recipe). Let’s call our function for the first time:
message = "Heisann"
result = split(message, "i") # Call our function
print(result)
('Hei', 'sann')
Function syntax#
The syntax is the following:
def functionname(arg1, arg2="default", arg3=1.0, ...):
"Docstring"
<block of statements>
return [expression]
We have a few options how to call a function:
functionname(1.0, "x", "i")
is the same as
functionname(arg1=1.0, arg2="x", arg2="i")
Default arguments can be left out:
functionname(1.0, args3="i")
Positional arguments must appear before keyword arguments:
functionname(arg3='i', "x") # invalid
Multiple return values#
Often it is useful to return multiple values in a function. This is achieved by packing the return values into a tuple:
def coordinates():
x = 1
y = 2
return x, y # Note: Short notation for tuple([x, y])
When calling the function, we can extract the two coordinate values from the tuple again:
xy = coordinates()
x = xy[0]
y = xy[1]
or we use the shorter notation:
x, y = coordinates() # Note: Python automatically "unpacks" the tuple entries