C.R.U.D. tutorial

In this tutorial we are going to add the concept of a notepad (title and body) to our application. The logical steps are detailed as a first approach to the development of uvlhub .

Create a new feature
1. Declare the feature
Model design
Inclusion of dependencies
Default route: list all my notepads
Migrations
1. Create a new migration
2. Apply the new migration
Design form
Complete C.R.U.D.
Fill in the tests

Every command below runs where the application runs.

Docker installation. Inside the application container. Either open a shell once:

docker exec -it web_app_container bash

or prefix each command with docker exec -it web_app_container.

Manual installation. From the root of the repository, with the virtual environment active and the project root on the import path, as the manual installation guide explains:

source venv/bin/activate
export PYTHONPATH=$(pwd)

Without PYTHONPATH every rosemary command, starting with the first one below, stops with ModuleNotFoundError: No module named 'app'; the flask db commands work either way.

The “No product config.py found” line

Every rosemary and flask db command prints No product config.py found for 'splent_app', using SPLENT default config. before its own output. It is a notice, not an error: the application has no product-level config.py and runs on the framework defaults.

Create a new feature

The unit of extension in uvlhub is a feature: a self-contained package under app/features/. We are going to create the notepad feature with the Rosemary CLI:

rosemary feature:create notepad

This creates app/features/notepad/ with the blueprint, the three layers, a form, a seeder, a template, an asset and six test files:

app/features/notepad/
├── __init__.py
├── forms.py
├── models.py
├── repositories.py
├── routes.py
├── seeders.py
├── services.py
├── assets/
│   └── js/
│       └── scripts.js
├── templates/
│   └── notepad/
│       └── index.html
└── tests/
    ├── __init__.py
    ├── locustfile.py
    ├── test_unit.py
    ├── test_repository.py
    ├── test_service.py
    ├── test_integration.py
    └── test_selenium.py

Take some time to examine each file and understand how they are related. The full breakdown of the scaffold is on the Create feature page.

How the script reaches the browser

Notice that templates/notepad/index.html has no <script> tag. The generated __init__.py declares assets/js/scripts.js with register_asset in init_feature, and base_template.html renders every registered script from one place. You never write a per-feature <script> tag; the asset route only serves the js, css and dist subfolders of assets/.

Declare the feature

Creating the folder is not enough. app/feature_loader.py only loads features declared in the root pyproject.toml, so add notepad to [tool.splent].features:

[tool.splent]
features = [
    "auth",
    "dataset",
    "explore",
    "featuremodel",
    "flamapy",
    "hubfile",
    "notepad",
    "profile",
    "public",
    "team",
    "zenodo",
]

Reboot required!

Flask registers blueprints once, at startup. Even with the feature declared, the routes do not exist until the server restarts. With Docker:
docker restart web_app_container
With a manual installation, stop flask run with Ctrl+C and start it again:
flask run --host=0.0.0.0 --reload --debug
--reload does not spare you this restart: the reloader only watches *.py files, so editing pyproject.toml never triggers one. The next edit to any .py file would, but do the restart now, so the routes exist before you continue. It is the only restart the tutorial needs: from here on, every change to models.py, routes.py and the templates is picked up on its own.

The proof that the restart worked is the server, not the CLI: http://localhost/notepad (http://localhost:5000/notepad on a manual installation) answered 404 before and answers 200 now, the application shell with an empty content area. Then list the routes of the feature:

rosemary route:list notepad

You should see something like this:

Listing routes for the 'notepad' module...
Endpoint          Methods    Route
notepad.assets    GET        /notepad/<subfolder>/<filename>
notepad.index     GET        /notepad

route:list builds its own copy of the application in a new process, so it lists the routes even before the restart. That is why the 404 to 200 check above is the one to trust.

Model design

Let’s make the Notepad model a bit more interesting. Let’s add two fields and an owner user.

The app/features/notepad/models.py file would look like this:

from app import db


class Notepad(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(256), nullable=False)
    body = db.Column(db.Text, nullable=False)
    user_id = db.Column(db.Integer, db.ForeignKey('user.id'), nullable=False)

    user = db.relationship('User', backref='notepads', lazy=True)

    def __repr__(self):
        return f'Notepad<{self.id}, Title={self.title}, Author={self.user.email}>'

Users are identified by email

The User model in app/features/auth/models.py has email, not username. Reaching for self.user.username raises AttributeError at the first repr().

Inclusion of dependencies

Since this is your first time developing this project, it can be a bit confusing to manage dependencies.

Before you continue, make sure that at the beginning of the routes.py file you have the following content:

from flask import abort, render_template, redirect, url_for, flash
from flask_login import login_required, current_user
from flask_wtf import FlaskForm
from splent_framework.utils.form_helpers import form_error, form_success

from app.features.notepad import notepad_bp
from app.features.notepad.forms import NotepadForm
from app.features.notepad.services import NotepadService

notepad_service = NotepadService()

form_error and form_success are small presentation helpers shipped by splent_framework. They do exactly what their names say, and nothing more:

def form_success(endpoint, message, category="success", **url_kwargs):
    flash(message, category)
    return redirect(url_for(endpoint, **url_kwargs))


def form_error(template, form, errors=None, **context):
    for field, messages in (errors or {}).items():
        for msg in messages:
            flash(f"{field}: {msg}", "error")
    return render_template(template, form=form, **context)

They live in the framework rather than on BaseService because they are HTTP concerns. Services return data, routes turn data into responses. app/features/profile/routes.py and app/features/auth/routes.py are the reference uses.

Default route: list all my notepads

It’s a bit boring to work only with code and not see anything, so let’s do something interesting! Let’s re-define the /notepad route to list the notepads created by me (even if we don’t have any yet).

Define the route in `routes.py`

'''
READ ALL
'''
@notepad_bp.route('/notepad', methods=['GET'])
@login_required
def index():
    form = NotepadForm()
    notepads = notepad_service.get_all_by_user(current_user.id)
    return render_template('notepad/index.html', notepads=notepads, form=form)

Define the template `app/features/notepad/templates/notepad/index.html`

{% extends "base_template.html" %}

{% block title %}View my notepads{% endblock %}

{% block content %}

{% with messages = get_flashed_messages(with_categories=true) %}
  {% if messages %}
    {% for category, message in messages %}
      <div class="alert alert-{{ category }}" role="alert">{{ message }}</div>
    {% endfor %}
  {% endif %}
{% endwith %}

<a href="{{ url_for('notepad.create_notepad') }}">New notepad</a>

{% if notepads %}
    <ul>
    {% for notepad in notepads %}
        <li>
            <strong><a href="{{ url_for('notepad.get_notepad', notepad_id=notepad.id) }}">{{ notepad.title }}</a></strong> - {{ notepad.body }}
            <a href="{{ url_for('notepad.edit_notepad', notepad_id=notepad.id) }}">Edit</a>
            <form method="POST" action="{{ url_for('notepad.delete_notepad', notepad_id=notepad.id) }}">
                {{ form.hidden_tag() }}
                <button type="submit">Delete</button>
            </form>
        </li>
    {% endfor %}
    </ul>
{% else %}
    <p>You have no notepads.</p>
{% endif %}

{% endblock %}

The get_flashed_messages block matters: base_template.html does not render flash messages for you. A template that never asks for them does not lose them either: they stay queued in the session and show up, out of context, on the next page that does render them. Every template of this feature therefore carries the block.

This template links forward

create_notepad, get_notepad, edit_notepad and delete_notepad are written further down, in the Complete C.R.U.D. section. Until they exist, url_for raises BuildError: Could not build url for endpoint 'notepad.create_notepad'. Either comment out the links for now, or write the four routes before you load the page.

Add new function in NotepadService

The app/features/notepad/services.py file should look like this:

from app.features.notepad.repositories import NotepadRepository
from splent_framework.services.BaseService import BaseService


class NotepadService(BaseService):
    def __init__(self):
        super().__init__(NotepadRepository())

    def get_all_by_user(self, user_id):
        return self.repository.get_all_by_user(user_id)

BaseService comes from the splent_framework package and gives you create, count, get_by_id, get_or_404, update and delete, all delegating to the repository. Anything beyond that is yours to write.

Add new function in NotepadRepository

The app/features/notepad/repositories.py file should look like this:

from app.features.notepad.models import Notepad
from splent_framework.repositories.BaseRepository import BaseRepository


class NotepadRepository(BaseRepository):
    def __init__(self):
        super().__init__(Notepad)

    def get_all_by_user(self, user_id):
        return self.get_by_column('user_id', user_id)

BaseRepository already ships get_by_column, so filtering by owner is one line. Write the query by hand only when the base class has nothing that fits.

We go to the /notepad route in the browser. Since we use the middleware @login_required, it is necessary to log in using a test user:

User: user1@example.com
Pass: 1234

If we access /notepad we notice that it gives error: an HTTP 500 whose page, because the server runs with --debug, is the Werkzeug debugger showing sqlalchemy.exc.ProgrammingError: (pymysql.err.ProgrammingError) (1146, "Table 'uvlhubdb.notepad' doesn't exist"). The same traceback is printed by the server (the docker logs of the container, or the terminal running flask run). Why do you think it gives error?

Migrations

Even if you define a model, it does not automatically exist in the database. You need to update the database, but don’t even think of creating a table by hand! No, that’s what migrations are for.

Concept of a migration

A migration is a software artefact that details how a database evolves, i.e. how it migrates from one state to another.

Create a new migration

Since we have a new entity in our model, in this case Notepad, it is necessary to create a new migration:

flask db migrate -m "create_notepad_model"

This creates a file in migrations/versions/XXXXXXXXX_create_notepad_model.py, with XXXXXXXXX being a unique alphanumeric revision identifier. Take your time to read this file: autogeneration is a good first draft, not an oracle.

Let’s go back to the /notepad route and see that it still gives an error. Why do you think it happens, if we have already created a new migration?

Apply the new migration

It is important to understand that the above command has only created the migration file, but we have not executed it yet. To run new migrations:

flask db upgrade

We go to the /notepad route and see that the database error is gone. Excellent! Unless you commented the links out, what you get instead is a different HTTP 500, werkzeug.routing.exceptions.BuildError: Could not build url for endpoint 'notepad.create_notepad': index.html links to the four routes that you write in the Complete C.R.U.D. section, and Flask refuses to render a link to a route that does not exist yet. It disappears as soon as those routes are in place.

Design form

We are going to design a form thanks to the Flask-WTForms package.

Flask-WTForms

Flask-WTForms is a Flask extension that allows you to manage and validate forms in an efficient and structured way within Flask web applications. It combines the simplicity of HTML forms with the advantages of server-side data validation, all in a simple and reusable way.

The app/features/notepad/forms.py file must have this content:

from flask_wtf import FlaskForm
from wtforms import StringField, TextAreaField, SubmitField
from wtforms.validators import DataRequired, Length


class NotepadForm(FlaskForm):
    title = StringField('Title', validators=[DataRequired(), Length(max=256)])
    body = TextAreaField('Body', validators=[DataRequired()])
    submit = SubmitField('Save notepad')

Complete C.R.U.D.

With all that we have learned and thanks to the form, we are ready to design a complete C.R.U.D.

Create a notepad

Route in `routes.py`

'''
CREATE
'''
@notepad_bp.route('/notepad/create', methods=['GET', 'POST'])
@login_required
def create_notepad():
    form = NotepadForm()
    if form.validate_on_submit():
        notepad_service.create(
            title=form.title.data,
            body=form.body.data,
            user_id=current_user.id,
        )
        return form_success('notepad.index', 'Notepad created successfully!')
    if form.errors:
        return form_error('notepad/create.html', form, form.errors)
    return render_template('notepad/create.html', form=form)

Template `app/features/notepad/templates/notepad/create.html`

{% extends "base_template.html" %}

{% block title %}Create notepad{% endblock %}

{% block content %}

{% with messages = get_flashed_messages(with_categories=true) %}
  {% if messages %}
    {% for category, message in messages %}
      <div class="alert alert-{{ category }}" role="alert">{{ message }}</div>
    {% endfor %}
  {% endif %}
{% endwith %}

<form method="POST" action="{{ url_for('notepad.create_notepad') }}">
    {{ form.hidden_tag() }}
    <div>
        {{ form.title.label }}<br>
        {{ form.title(size=32) }}
    </div>
    <div>
        {{ form.body.label }}<br>
        {{ form.body(rows=5) }}
    </div>
    <div>
        {{ form.submit() }}
    </div>
</form>

{% endblock %}

Read a notepad

Route in `routes.py`

'''
READ BY ID
'''
@notepad_bp.route('/notepad/<int:notepad_id>', methods=['GET'])
@login_required
def get_notepad(notepad_id):
    notepad = notepad_service.get_or_404(notepad_id)

    if notepad.user_id != current_user.id:
        flash('You are not authorized to view this notepad', 'error')
        return redirect(url_for('notepad.index'))

    return render_template('notepad/show.html', notepad=notepad)

get_or_404 is inherited from BaseService, which delegates to BaseRepository.get_or_404. An unknown id aborts with a 404 before your ownership check ever runs.

Template `app/features/notepad/templates/notepad/show.html`

{% extends "base_template.html" %}

{% block title %}Notepad details{% endblock %}

{% block content %}

<h1>{{ notepad.title }}</h1>
<p>{{ notepad.body }}</p>
<a href="{{ url_for('notepad.index') }}">Back to Notepads</a>
{% endblock %}

Edit a notepad

Route in `routes.py`

'''
EDIT
'''
@notepad_bp.route('/notepad/edit/<int:notepad_id>', methods=['GET', 'POST'])
@login_required
def edit_notepad(notepad_id):
    notepad = notepad_service.get_or_404(notepad_id)
    if notepad.user_id != current_user.id:
        flash('You are not authorized to edit this notepad', 'error')
        return redirect(url_for('notepad.index'))

    form = NotepadForm(obj=notepad)
    if form.validate_on_submit():
        notepad_service.update(
            notepad_id,
            title=form.title.data,
            body=form.body.data,
        )
        return form_success('notepad.index', 'Notepad updated successfully!')
    if form.errors:
        return form_error('notepad/edit.html', form, form.errors, notepad=notepad)
    return render_template('notepad/edit.html', form=form, notepad=notepad)

Template `app/features/notepad/templates/notepad/edit.html`

{% extends "base_template.html" %}

{% block title %}Edit notepad{% endblock %}

{% block content %}

{% with messages = get_flashed_messages(with_categories=true) %}
  {% if messages %}
    {% for category, message in messages %}
      <div class="alert alert-{{ category }}" role="alert">{{ message }}</div>
    {% endfor %}
  {% endif %}
{% endwith %}

<form method="POST" action="{{ url_for('notepad.edit_notepad', notepad_id=notepad.id) }}">
    {{ form.hidden_tag() }}
    <div>
        {{ form.title.label }}<br>
        {{ form.title(size=32) }}
    </div>
    <div>
        {{ form.body.label }}<br>
        {{ form.body(rows=5) }}
    </div>
    <div>
        {{ form.submit() }}
    </div>
</form>

{% endblock %}

Delete a notepad

Route in `routes.py`

'''
DELETE
'''
@notepad_bp.route('/notepad/delete/<int:notepad_id>', methods=['POST'])
@login_required
def delete_notepad(notepad_id):
    if not FlaskForm().validate_on_submit():
        abort(400)
    notepad = notepad_service.get_or_404(notepad_id)
    if notepad.user_id != current_user.id:
        flash('You are not authorized to delete this notepad', 'error')
        return redirect(url_for('notepad.index'))

    if notepad_service.delete(notepad_id):
        return form_success('notepad.index', 'Notepad deleted successfully!')

    return form_success('notepad.index', 'Error deleting notepad', category='error')

BaseRepository.delete returns True when a row was removed and False when the id did not match anything, which is why the return value is worth branching on.

The delete form in index.html carries {{ form.hidden_tag() }}, but a token nobody checks protects nothing: the application has no global CSRF protection, so only validate_on_submit() verifies it. A bare FlaskForm has no fields, so validating it checks exactly the token, and a POST without one is refused with 400. The edit and create routes get the same check for free from form.validate_on_submit().

Take the time to check that everything is working properly. Try creating a notepad in the /notepad/create route.

You can list the routes again to see that they have been updated:

rosemary route:list notepad

Fill in the tests

feature:create already handed you app/features/notepad/tests/, with one file per layer of the testing pyramid. The five test_*.py files are each pre-tagged with their marker at module level; locustfile.py is driven by rosemary locust rather than pytest. They are stubs, and now you have something worth asserting on.

Start with tests/test_integration.py. The generated stub asserts that GET /notepad returns 200, which stopped being true the moment you added @login_required — the route now answers with a redirect to the login page. Make the test say so:

import pytest

pytestmark = pytest.mark.integration


def test_notepad_index_requires_login(test_client):
    response = test_client.get("/notepad", follow_redirects=False)
    assert response.status_code in (302, 303)
    assert "/login" in response.headers["Location"]

Then tests/test_service.py, where the database is real but there is no HTTP:

import pytest

from app.features.auth.repositories import UserRepository
from app.features.notepad.services import NotepadService

pytestmark = pytest.mark.service


def test_get_all_by_user_only_returns_own_notepads(test_app):
    with test_app.app_context():
        service = NotepadService()
        mine = UserRepository().create(email="mine@example.com", password="secret")
        theirs = UserRepository().create(email="theirs@example.com", password="secret")

        service.create(title="Mine", body="...", user_id=mine.id)
        service.create(title="Theirs", body="...", user_id=theirs.id)

        notepads = service.get_all_by_user(mine.id)

        assert [n.title for n in notepads] == ["Mine"]

The test_client and test_app fixtures come from splent_framework.fixtures.fixtures and are re-exported by the root conftest.py, so you never import them yourself.

Run the fast layers for your feature:

rosemary test notepad

That runs the unit, repository, service and integration markers. Add --e2e for the browser layer once you have written test_selenium.py against the Selenium grid, and use rosemary locust for the load layer.

Happy development!

C.R.U.D. tutorial

Table of contents

Create a new feature

Declare the feature

Model design

Inclusion of dependencies

Default route: list all my notepads

Define the route in routes.py

Define the template app/features/notepad/templates/notepad/index.html

Add new function in NotepadService

Add new function in NotepadRepository

Migrations

Create a new migration

Apply the new migration

Design form

Complete C.R.U.D.

Create a notepad

Route in routes.py

Template app/features/notepad/templates/notepad/create.html

Read a notepad

Route in routes.py

Template app/features/notepad/templates/notepad/show.html

Edit a notepad

Route in routes.py

Template app/features/notepad/templates/notepad/edit.html

Delete a notepad

Route in routes.py

Fill in the tests

Define the route in `routes.py`

Define the template `app/features/notepad/templates/notepad/index.html`

Route in `routes.py`

Template `app/features/notepad/templates/notepad/create.html`

Route in `routes.py`

Template `app/features/notepad/templates/notepad/show.html`

Route in `routes.py`

Template `app/features/notepad/templates/notepad/edit.html`

Route in `routes.py`