Introduction to web servers

Introduction to web servers#

Tasks of a web server#

A web server

The Hypertext Transfer Protocol#

The Hypertext Transfer Protocol (HTTP) is designed to enable communications between clients and servers.

It is a request-response protocol between a client and server.

HTTP Request Methods: GET and POST#

Common HTTP methods:

GET: Requests data from a server
POST: Submits data to be processed by the server (Create/Modify)
PUT: Submits data to be processed by the server (Upload)
DELETE: Delete data on the server

Live demo of a GET request#

python3 -m pip install "fastapi[all]"

python3 hello_world_web.py    # Start a simple web server

import requests

r = requests.get("http://127.0.0.1:8000")
r.text

'"Hello World!"'

A(n extremely incomplete list of) popular scripting webservers#

FastAPI (Python)
Flask (Python)
Django (Python)
Tornado (Python)
Express (Node.js)
Ruby on Rails (Ruby)

Building a webserver with FastAPI#

What is FastAPI?#

FastAPI is a web application framework built for making web APIs.

It’s based strongly on the idea of type hints.

You can use FastAPI to:

build a website (blog, private homepage)
build an API server

Some key FastAPI features:

Development server and debugger
Integrated support for unit testing
Open-source
Deep integration with typing tools for validation, better errors

Documentation#

https://fastapi.tiangolo.com

Installation#

conda install fastapi uvicorn

or

python3 -m pip install "fastapi[all]"

Hello world in FastAPI#

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def hello():
    return "Hello World!"

Normally, your app script will end with something like:

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="127.0.0.1", port=8000)

but we’re going to do something special to allow running in the notebook:

import sys
from threading import Thread

import uvicorn


def run_app():
    sys.stderr = sys.stdout
    uvicorn.run(app, host="127.0.0.1", port=8000)


app_thread = Thread(target=run_app)
app_thread.start()

INFO:     Started server process [23025]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Or we can run the server with:

python3 hello_world.py

    INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Then open http://127.0.0.1:8000 in your web browser. To stop the server, hit Control-C.

import requests

r = requests.get("http://127.0.0.1:8000")
r.text

INFO:     127.0.0.1:62653 - "GET / HTTP/1.1" 200 OK

'"Hello World!"'

What happened?#

from fastapi import FastAPI

app = FastAPI()

The instance of the FastAPI class is our web application.

The first argument is needed so that FastAPI knows where to look for templates, static files, and so on.

@app.get("/")
def hello():
    return "Hello World!"

We then use the get() decorator to tell Flask what URL should trigger our function, and wht HTTP method. By default, FastAPI answers to GET requests, and the return value of the function is the answer of the GET request which will be serialized to JSON

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="127.0.0.1", port=8000)

Note: You need to restart the sever when changing your code.

Debug mode#

You can also run our app in debug mode with:

uvicorn hello_world_web:app --debug 

where:

hello_world_web is the name of our hello_world_web.py module, and
app is the name we used for our FastAPI object inside
--debug tells the server (uvicorn) to do a few things to help debugging and development:
- log with more detail
- reload the server when it notices changes to files (so you don’t have to stop and restart the server)

Making the server available in your network#

If you want your server to be accessible from your entire network, you need to launch the server with:

uvicorn.run(app, host='0.0.0.0')

or from the commandline

uvicorn hello_world_web:app --host 0.0.0.0

Important: This allows anyone in your network to access your server, which might be a severe security risk!

Adding more URLs#

We can serve additional URLs by adding new functions with the decorator for each HTTP verb (usually @app.get).

users = {
    "spike": "Spike Spiegel",
    "jet": "Jet Black",
    "faye": "Faye Valentine",
    "ed": "Edward Wong",
}


@app.get("/users")
def show_user_overview():
    return users


# uvicorn.run(app, port=8000)

Matches http://localhost:8000/users

r = requests.get("http://localhost:8000/users")
r

INFO:     127.0.0.1:63232 - "GET /users HTTP/1.1" 200 OK

200 http://localhost:8000/users
headers:   content-length: 86
  content-type: application/json
  date: Wed, 08 Nov 2023 11:26:46 GMT
  server: uvicorn

body (application/json):
{'spike': 'Spike Spiegel',
 'jet': 'Jet Black',
 'faye': 'Faye Valentine',
 'ed': 'Edward Wong'}

r.headers["Content-Type"]

'application/json'

r.json()

{'spike': 'Spike Spiegel',
 'jet': 'Jet Black',
 'faye': 'Faye Valentine',
 'ed': 'Edward Wong'}

r.json()["ed"]

'Edward Wong'

@app.get("/users/{username}")
def show_user_profile(username):
    # show the user profile for that user
    return {
        "username": username,
        "full name": users[username],
    }

Matches http://localhost:8000/users/ed or http://localhost:8000/users/NAME for any NAME in our user list.

r = requests.get("http://localhost:8000/users/ed")
r.json()

INFO:     127.0.0.1:63719 - "GET /users/ed HTTP/1.1" 200 OK

{'username': 'ed', 'full name': 'Edward Wong'}

What happens if we request a name that doesn’t exist?

r = requests.get("http://localhost:8000/users/min")
r

INFO:     127.0.0.1:63831 - "GET /users/min HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/minrk/conda/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/Users/minrk/conda/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/Users/minrk/conda/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/Users/minrk/conda/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/Users/minrk/conda/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
  File "/Users/minrk/conda/lib/python3.10/site-packages/fastapi/routing.py", line 193, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/Users/minrk/conda/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/Users/minrk/conda/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/Users/minrk/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/Users/minrk/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/ipykernel_23025/4191858267.py", line 6, in show_user_profile
    "full name": users[username],
KeyError: 'min'

500 http://localhost:8000/users/min
headers:   content-length: 21
  content-type: text/plain; charset=utf-8
  date: Wed, 08 Nov 2023 11:30:00 GMT
  server: uvicorn

body (text/plain; charset=utf-8):
Internal Server Error

That brings us to

Error handling#

HTTP errors are common. You are probably familiar with “404 not found”

Every HTTP response has a numerical status, in the range 100-599.

Each 100 range has a broad category, and then there are more specific codes within those ranges:

1XX is for protocol-level information (don’t worry about it)
2XX is for success
3XX is for redirection (maybe okay, retry somewhere else)
4XX is for client error (your fault)
5XX is for server error (my fault)

A great resource is https://developer.mozilla.org/en-US/docs/Web/HTTP/Status, which lists all the common and standardized codes, and what they are for.

If the client request was wrong for some reason, it can respond with 400: Bad Request. That’s not very informative.

A common mistake is for a request to be invalid because it asked for smething that doesn’t exist. There’s a special 4XX code for that: 404: not found.

That’s the error we should raise when someone asks for a username that doesn’t exist.

from fastapi import HTTPException, status


@app.get("/users2/{username}")
def show_user_profile(username):
    # show the user profile for that user
    if username not in users:
        raise HTTPException(
            status.HTTP_404_NOT_FOUND,
            detail=f"No such user: {username}",
        )
    return {
        "username": username,
        "full name": users[username],
    }

Now let’s try again

r = requests.get("http://localhost:8000/users2/vicious")
r

INFO:     127.0.0.1:64317 - "GET /users2/vicious HTTP/1.1" 404 Not Found

404 http://localhost:8000/users2/vicious
headers:   content-length: 34
  content-type: application/json
  date: Wed, 08 Nov 2023 11:32:38 GMT
  server: uvicorn

body (application/json):
{'detail': 'No such user: vicious'}

r.status_code

r.json()

{'detail': 'No such user: vicious'}

Templates#

So far our webserver only served JSON, but no HTML documents.

The first thing you need to do is declare a response_class

from fastapi.responses import HTMLResponse

@app.get("/login", response_class=HTMLResponse)

That tells FastAPI to respond with HTML as text, rather than serialize a JSON model.

One solution would be to define the entire HTML string in the URL handler, e.g.:

@app.get('/plainhtml', respose_type=HTMLResponse)
def some_html():
    return '''
<html>
<header><title>The title</title></header>
<body>
Hello world
</body>
</html>    
    '''

However, it is more common to use a templating system (Jinja is the most popular one) that makes our life a lot easier:

from fastapi import Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates

templates = Jinja2Templates(directory="templates")


@app.get("/post/", response_class=HTMLResponse)
@app.get("/post/{name}", response_class=HTMLResponse)
def post(request: Request, name=None):
    return templates.TemplateResponse(
        "post.html",
        {
            "request": request,
            "name": name,
        },
    )

Jinja will look for templates in the templates folder.

<!-- ./templates/post.html -->

<!doctype html>
<title>Hello from FastAPI</title>
{% if name %}
  <h1>Displaying blog post {{ name }}!</h1>
{% else %}
  <h1>No post name given!</h1>
{% endif %}

Matches http://localhost:8000/post/History Of Science

from IPython.display import HTML

r = requests.get("http://localhost:8000/post/History%20Of%20Science")
HTML(r.text)

INFO:     127.0.0.1:65019 - "GET /post/History%20Of%20Science HTTP/1.1" 200 OK

Hello from FastAPI

Displaying blog post History Of Science!

INFO:     127.0.0.1:65065 - "GET /post/History%20Of%20Science HTTP/1.1" 200 OK

Interactive web applications with HTML forms#

Using the template, we can now create a HTML form with a POST request

<!-- ./templates/login.html -->

<!doctype html>
<title>Login</title>

{% if error %}
<p style="color:red">{{ error }}</p>
{% endif %}

<form action="login" method="POST">
    Username:
    <br>
    <input type="text" name="username">
    <br>
    Password:
    <br>
    <input type="password" name="password">
    <br>
    <input type="submit" value="Submit">
</form>

@app.get("/login")
def login(request: Request):
    return templates.TemplateResponse("login.html", {"request": request})

Matches http://localhost:8000/login

r = requests.get("http://localhost:8000/login")
HTML(r.text)

INFO:     127.0.0.1:65243 - "GET /login HTTP/1.1" 200 OK

Login

Handling the `POST` request.#

The form above sends a POST request to the handle_login URL.

We can use

@app.post('/login')

to create a new handler that accepts POST requests.

We use Form(...) to declare inputs to the request that should come the login form fields. The names of these variables are important! They must match

from fastapi import Form


@app.post("/login")
def login(username=Form(...), password=Form(...)):
    if username == "ein" and password == "datadog":
        return {"username": username}
    else:
        raise HTTPException(status.HTTP_403_FORBIDDEN)

INFO:     127.0.0.1:65506 - "GET /login HTTP/1.1" 200 OK
INFO:     127.0.0.1:65528 - "POST /login HTTP/1.1" 403 Forbidden
INFO:     127.0.0.1:49173 - "POST /login HTTP/1.1" 200 OK

Try it on http://localhost:8000/login