dataset

The dataset feature is the center of uvlhub. It owns the dataset lifecycle end to end: the upload form and its staging area, persistence of a dataset with its metadata and authors, the push to Zenodo that mints the DOI, the public DOI landing page, the zipped download, per-dataset view and download metrics, and the REST API at /api/v1/datasets/. It lives in app/features/dataset/.

Table of contents

  1. What it does
  2. Routes
  3. REST API
  4. Models
  5. Services and repositories
  6. Dependencies
  7. Templates and assets
  8. Tests
  9. Configuration

What it does

A dataset is a set of UVL feature models uploaded together under one metadata record. The flow is:

  1. The logged-in user opens /dataset/upload. Each UVL file dropped on the page is POSTed to /dataset/file/upload and staged in the user’s temp folder; a staged file can be removed again through /dataset/file/delete.
  2. Submitting the form POSTs to /dataset/upload. DataSetService.upload_dataset persists the dataset, its metadata, authors, feature models and file rows locally, moves the staged files under uploads/user_<id>/dataset_<id>/, and then pushes everything to Zenodo. The Zenodo half is best-effort: if it fails, the dataset stays in the hub without a DOI and the call answers {"status": "partial"} instead of an error (see Zenodo).
  3. Once a DOI is stored, the dataset is synchronized and its landing page is served at /doi/<doi>/ to anyone, no login required.

Routes

Real output of rosemary route:list for this blueprint:

Endpoint                             Methods                    Route
dataset.assets                       GET                        /dataset/<subfolder>/<filename>
dataset.create_dataset               GET                        /dataset/upload
dataset.dataset                      DELETE, GET, POST, PUT     /api/v1/datasets/<int:id>
dataset.datasets                     DELETE, GET, POST, PUT     /api/v1/datasets/
dataset.delete                       POST                       /dataset/file/delete
dataset.download_dataset             GET                        /dataset/download/<int:dataset_id>
dataset.get_unsynchronized_dataset   GET                        /dataset/unsynchronized/<int:dataset_id>/
dataset.list_dataset                 GET, POST                  /dataset/list
dataset.subdomain_index              GET                        /doi/<path:doi>/
dataset.upload                       POST                       /dataset/file/upload
dataset.upload_dataset               POST                       /dataset/upload
Route Auth What it does
/dataset/upload (GET) login Renders the upload form.
/dataset/upload (POST) login Validates the form and runs DataSetService.upload_dataset.
/dataset/file/upload login Stages one UVL file in the user’s temp folder. Rejects anything that is not .uvl; colliding names get a " (n)" suffix.
/dataset/file/delete login Removes a staged file. Anonymous requests are redirected to the login form.
/dataset/list login The user’s synchronized and unsynchronized datasets, newest first.
/dataset/download/<id> public Zips the dataset’s files and serves the archive.
/doi/<doi>/ public DOI landing page. Old DOIs recorded in DOIMapping are 302-redirected to the new one; unknown DOIs answer 404.
/dataset/unsynchronized/<id>/ login Detail page for a dataset that has no DOI yet, visible only to its owner.
/dataset/<subfolder>/<filename> public The feature’s static assets.

Two details of /dataset/download/<id> are deliberate. The download record is written only after the file response has been built, so a request that 404s never counts as a download. And the record is deduplicated per (user, dataset, cookie): the route reuses the visitor’s download_cookie if one is already present and only sets it when absent, so repeated downloads from the same browser count once. The cookie carries a two-year lifetime, matching hubfile’s, so the dedupe survives browser restarts. The DOI landing page applies the same idea to views through a view_cookie.

REST API

app/features/dataset/api.py registers a Flask-RESTful resource on the blueprint at /api/v1/datasets/ and /api/v1/datasets/<int:id>. The resource is generated by splent_framework’s create_resource(DataSet, dataset_serializer), a generic CRUD resource — that is why route:list shows DELETE, GET, POST, PUT on both rules. In practice the read side is the useful one; the serializer exposes dataset_id, created, name, doi and the nested files with their human-readable size:

curl http://localhost/api/v1/datasets/
curl http://localhost/api/v1/datasets/1

{
    "dataset_id": 1,
    "created": "2026-07-21T12:32:06",
    "name": "Sample dataset 1",
    "doi": "http://localhost/doi/10.1234/dataset1",
    "files": [
        {"file_id": 1, "file_name": "file1.uvl", "size": "414 bytes"}
    ]
}

Models

app/features/dataset/models.py defines:

Model Table Purpose
DataSet data_set One dataset. FK user_id to auth’s user (not nullable), FK ds_meta_data_id. Has the feature_models relationship.
DSMetaData ds_meta_data Title, description, publication type, tags, publication_doi, dataset_doi, Zenodo deposition_id.
Author author An author row. Carries two FKs: ds_meta_data_id into this feature and fm_meta_data_id into featuremodel’s fm_meta_data table.
DSMetrics ds_metrics Aggregate numbers attached to DSMetaData.
DSDownloadRecord ds_download_record One download, with nullable user_id and the deduplication cookie.
DSViewRecord ds_view_record One view, same shape.
DOIMapping doi_mapping Maps a superseded DOI to its replacement for /doi/<doi>/ redirects.

PublicationType is the enum behind the publication type of both datasets and feature models — featuremodel imports it from here.

Two DataSet helpers are worth knowing. get_zenodo_url() returns the Zenodo record URL only when the metadata has both a dataset_doi and a deposition_id, and None otherwise, so templates and to_dict() can render unsynchronized datasets without a broken link. And to_dict() guards tags before splitting, so a NULL tags column serializes as an empty list; the detail template applies the same guard before iterating.

Services and repositories

app/features/dataset/services.py:

Service Purpose
DataSetService The orchestrator: create_from_form (one transaction for metadata, authors, feature models and file rows), move_feature_models, save_temp_uvl, delete_temp_file, upload_dataset (local persist, then best-effort Zenodo), build_download_archive, get_uvlhub_doi, plus the synchronized/unsynchronized lookups and the statistics counts.
DSDownloadRecordService record_download, deduplicated per (user, dataset, cookie).
DSViewRecordService create_cookie, which records a view once per cookie.
DSMetaDataService Metadata updates and filter_by_doi.
DOIMappingService Resolves old DOIs to new ones.
AuthorService Thin CRUD wrapper.
SizeService get_human_readable_size, the bytes-to-KB/MB/GB formatter the whole app uses.

build_download_archive resolves the dataset’s upload folder under WORKING_DIR (<WORKING_DIR>/uploads/user_<id>/dataset_<id>) and writes the zip into a fresh temp directory, so the archive path never depends on the process’ current working directory.

app/features/dataset/repositories.py holds one repository per model, all extending BaseRepository. The statistics the public landing page shows (count_synchronized_datasets, total_dataset_downloads, total_dataset_views) are computed in SQL with COUNT, not by loading rows into Python and counting them.

Dependencies

Measured over production code only (tests excluded), dataset imports four other features:

Imports Where Why
auth.services services.py AuthenticationService resolves the current user’s temp folder in move_feature_models.
featuremodel.repositories services.py DataSetService instantiates FeatureModelRepository and FMMetaDataRepository directly. This is a layer violation: a service reaching into another feature’s repositories, skipping FeatureModelService.
hubfile.repositories services.py Same pattern with HubfileRepository and the record repositories — the same layer violation.
zenodo.services services.py, lazily ZenodoService is imported inside upload_dataset, not at module level. Removing zenodo therefore does not break importing dataset; it breaks the upload at call time instead.

The seeder (seeders.py) additionally imports auth.models, featuremodel.models and hubfile.models to build the sample data.

dataset is also the most depended-on feature in the codebase. Six features import it: explore (queries its models to search), featuremodel (imports Author and PublicationType at module level in its models.py), hubfile (models, repositories and services), profile (uses DataSetRepository for the user’s dataset list), public (landing-page statistics via DataSetService) and zenodo (takes DataSet objects as input).

There is also coupling that no import graph shows: the templates. view_dataset.html builds URLs into two other blueprints with url_for — four references into flamapy (flamapy.valid, flamapy.to_glencoe, flamapy.to_cnf, flamapy.to_splot) and one into hubfile (hubfile.download_file):

<a class="dropdown-item" href="{{ url_for('flamapy.valid', file_id=file.id) }}">SAT validity check</a>
<a class="dropdown-item" href="{{ url_for('hubfile.download_file', file_id=file.id) }}">

Disabling flamapy in [tool.splent] breaks the dataset detail page. With the flamapy blueprint unregistered, the first url_for('flamapy...') in view_dataset.html raises a BuildError and the DOI landing page answers 500. The [tool.splent] feature lists make flamapy look optional; this template dependency means that, for dataset, it is not.

The template’s JavaScript also fetches /file/view/<id> and /file/download/<id> (hubfile routes) by literal path.

Taken together: dataset, featuremodel and hubfile form a module-level dependency triangle — each of the three imports the other two. Together with auth they are the product’s core, and they are only meaningfully deployable as a unit.

Templates and assets 

app/features/dataset/
├── templates/dataset/
│   ├── list_datasets.html     # /dataset/list
│   ├── upload_dataset.html    # /dataset/upload
│   └── view_dataset.html      # /doi/<doi>/ and /dataset/unsynchronized/<id>/
├── assets/js/scripts.js       # upload form: author blocks, staging uploads, submit
└── uvl_examples/              # file1.uvl … file12.uvl, copied by the seeder

scripts.js is registered through the framework asset registry in init_feature and drives the upload page: dynamic author sub-forms, staging files to /dataset/file/upload, and submitting the whole form to /dataset/upload. view_dataset.html additionally loads Pyodide to run flamapy checks client-side.

The seeder (DataSetSeeder, priority 2, after auth’s users) creates four sample datasets with twelve feature models and copies the uvl_examples files into uploads/.

Tests

One file per layer, marker set at module level:

app/features/dataset/tests/
├── test_unit.py          # pytest.mark.unit
├── test_repository.py    # pytest.mark.repository
├── test_service.py       # pytest.mark.service
├── test_integration.py   # pytest.mark.integration
├── test_selenium.py      # pytest.mark.e2e
└── locustfile.py         # load tests

The suite pins the behaviors described above: get_zenodo_url returning None without a deposition, the DOI landing page rendering a dataset without tags, temp-file upload/delete and the login redirect on /dataset/file/delete, build_download_archive zipping the upload folder, and download records deduplicated per cookie. Run one layer at a time:

rosemary test dataset --unit
rosemary test dataset --integration
rosemary test dataset --e2e

Configuration

grep os.getenv over the feature’s production code finds two variables:

Variable Read in Used for
WORKING_DIR services.py (twice), seeders.py Root under which uploads/user_<id>/dataset_<id>/ lives; also where the seeder finds uvl_examples.
DOMAIN services.py Builds the public DOI URL, http://<DOMAIN>/doi/<doi> (default localhost).

The feature ships no .env.example of its own; both variables come from the root .env. The Zenodo credentials used during upload belong to the zenodo feature.