mafw.devtools.documentation
Documentation management utilities for MAFw development tools.
This package provides business logic for building versioned documentation, managing versions.json, generating redirect pages, creating requirements documentation, and running local documentation servers.
Functions
|
Create worktree for tag, run sphinx-build, save log. |
|
Build a GitLab API configuration from provided values and environment context. |
|
Build authentication headers for GitLab API requests. |
|
Create worktree for tag, run sphinx-build with latex builder, then make PDF. |
|
Check if multiversion structure exists (other version directories). |
|
Copy patch files needed for older versions. |
|
Create a zip archive for a built documentation version directory. |
Download a documentation zip archive from the GitLab Generic Package Registry. |
|
Ensure that the Sphinx Python package is available. |
|
|
Ensure versions.json exists in outdir. |
|
Extract a documentation zip archive into the repository root and remove the zip. |
|
Keep only the latest micro version per minor (major.minor). |
|
Filter tags based on a regular expression pattern. |
|
Filter versions within an inclusive semantic-version range. |
|
Find the repository root directory. |
|
Format bytes to human-readable size. |
Format a timestamp for server metadata. |
|
|
Generate an HTML page listing all available PDFs. |
|
Generate the RST substitution file for the supported Python range. |
|
Generate an RST file with the dependencies of a given group. |
Get the name of the currently checked out branch. |
|
|
Calculate total size of a directory in bytes. |
|
Return list of (Version, tag) tuples sorted ascending. |
|
Get the git revision hash for a given reference. |
|
Probe whether the server responds at the given address:port. |
|
Return True if commit a is ancestor of commit b (git merge-base --is-ancestor). |
|
List local mafw-docs zip files in a directory. |
|
List generic packages for mafw-docs in the GitLab Package Registry. |
|
Mirror a version directory from one tag to another. |
|
Normalize a registry item into (version, file_name). |
|
Parse and validate a mafw-docs zip filename. |
|
Parse Sphinx build log to extract warning and error counts, and warning messages. |
|
Parse vX.Y.Z(.W) into tuple of ints for sorting. |
|
Check whether a process resembles the expected http.server invocation. |
|
Remove oldest version directories until total size is below threshold. |
|
Read local documentation server metadata if available. |
Regenerate versions.json after pruning, excluding removed versions. |
|
|
Report build status with warning/error summary. |
Resolve package IDs for mafw-docs generic packages by version. |
|
|
Helper to run commands with consistent behavior. |
Return the default directory served by the local documentation server. |
|
|
Return the metadata path associated to a served directory. |
|
Sort tags using semantic versioning comparison. |
Upload a documentation zip archive to the GitLab Generic Package Registry. |
|
|
Create a legacy redirect page at the root of the output directory. |
|
Create a redirect page for a version alias. |
|
Create a _redirects file for GitLab Pages. |
|
Create a landing page for the project root with links to documentation and coverage. |
|
Write local documentation server metadata. |
|
Write versions information to a JSON file. |
Classes
|
Possible status values for the multiversion-doc local documentation server. |
- class mafw.devtools.documentation.ServerStatusEnum(*values)
Bases:
StrEnumPossible status values for the multiversion-doc local documentation server.
- Variables:
running – The server process exists, matches the expected signature and responds over HTTP.
stale – The server process exists and matches the expected signature, but does not respond over HTTP.
stopped – The recorded PID does not exist or does not match the expected signature.
unknown – The metadata file is missing, so no status can be determined.
- static _generate_next_value_(name, start, count, last_values)
Return the lower-cased version of the member name.
- mafw.devtools.documentation._load_supported_python_versions(*, repo_root: Path | None = None) list[str]
Load the supported Python versions declared in
pyproject.toml.- Parameters:
repo_root (Path | None) – Repository root directory, defaults to auto-detection
- Returns:
Sorted list of supported
major.minorversions.- Return type:
list[str]
- mafw.devtools.documentation.build_for_tag(tag: str, outdir: Path, tmproot: Path, use_latest_conf: bool = False, keep_tmp: bool = False) Tuple[bool, str]
Create worktree for tag, run sphinx-build, save log.
- Parameters:
tag (str) – Git tag to build documentation for
outdir (Path) – Output directory for built documentation
tmproot (Path) – Root temporary directory
use_latest_conf (bool) – Whether to use latest conf.py, defaults to False
keep_tmp (bool) – Whether to keep temporary files, defaults to False
- Returns:
Tuple of (success, log_contents)
- Return type:
Tuple[bool, str]
- mafw.devtools.documentation.build_gitlab_api_configuration(api_url: str | None, project_id: int | None, token: str | None) GitlabAPIConfiguration
Build a GitLab API configuration from provided values and environment context.
- Parameters:
api_url (str | None) – Optional override for the GitLab API URL.
project_id (int | None) – Optional override for the GitLab project ID.
token (str | None) – Optional override for the authentication token.
- Returns:
The validated GitLab API configuration.
- Return type:
- Raises:
ValueError – If any required configuration value is missing.
- mafw.devtools.documentation.build_gitlab_auth_headers(api_config: GitlabAPIConfiguration) MutableMapping[str, str | bytes]
Build authentication headers for GitLab API requests.
- Parameters:
api_config (GitlabAPIConfiguration) – The GitLab API configuration.
- Returns:
A mutable mapping of HTTP headers.
- Return type:
MutableMapping[str, str | bytes]
- mafw.devtools.documentation.build_pdf_for_tag(tag: str, html_tag_dir: Path, tmproot: Path, use_latest_conf: bool = False, keep_tmp: bool = False) Tuple[bool, str, Path | None]
Create worktree for tag, run sphinx-build with latex builder, then make PDF.
- Parameters:
tag (str) – Git tag to build PDF for
html_tag_dir (Path) – Directory containing HTML output for the tag
tmproot (Path) – Root temporary directory
use_latest_conf (bool) – Whether to use latest conf.py, defaults to False
keep_tmp (bool) – Whether to keep temporary files, defaults to False
- Returns:
Tuple of (success, log_contents, pdf_path)
- Return type:
Tuple[bool, str, Path | None]
- mafw.devtools.documentation.check_multiversion_structure(outdir: Path) bool
Check if multiversion structure exists (other version directories).
- Parameters:
outdir (Path) – Output directory to check
- Returns:
True if other versions exist
- Return type:
bool
- mafw.devtools.documentation.copy_patch_files(docs_src: Path) None
Copy patch files needed for older versions.
- Parameters:
docs_src (Path) – Path to documentation source directory
- mafw.devtools.documentation.create_docs_zip_for_tag(outdir: Path, tag: str, zip_filepath: Path) Path
Create a zip archive for a built documentation version directory.
The built docs are expected under
outdir / tag(e.g.docs/build/doc/vX.Y.Z). The produced zip is laid out so that extracting it from the repository root recreates the original directory structure (e.g.docs/build/doc/vX.Y.Z/...).The zip file is created at
zip_filepath / f"mafw-docs-{tag}.zip".Notes
Symlinks are skipped to avoid ambiguous extraction behavior across platforms.
- param outdir:
Output directory that contains the built docs
- type outdir:
Path
- param tag:
Version tag (e.g.
v2.1.0)- type tag:
str
- param zip_filepath:
Directory where the zip file is written
- type zip_filepath:
Path
- return:
Path to the created zip archive
- rtype:
Path
- raises FileNotFoundError:
If the built docs directory does not exist
- raises NotADirectoryError:
If the built docs path is not a directory
- mafw.devtools.documentation.download_docs_zip_from_gitlab_generic_registry(api_config: GitlabAPIConfiguration, package_version: str, download_dir: Path, package_name: str = 'mafw-docs', file_name: str | None = None, timeout_s: float = 60.0) Path | None
Download a documentation zip archive from the GitLab Generic Package Registry.
The equivalent GitLab API endpoint is:
GET /projects/:id/packages/generic/:package_name/:package_version/:file_nameThe function first issues a HEAD request to determine if the file exists: - 404: the package file does not exist (cache miss) and
Noneis returned. - 200: the file exists and is downloaded. - otherwise: a warning is printed andNoneis returned.Authentication headers: -
JOB-TOKENwhen running on CI (api_config.on_ciis True) -PRIVATE-TOKENfor local execution- Parameters:
api_config (GitlabAPIConfiguration) – GitLab API configuration
package_version (str) – Package version (typically the git tag, e.g.
v2.1.0)download_dir (Path) – Directory where the downloaded zip is stored
package_name (str) – Generic package name, defaults to
mafw-docsfile_name (str | None) – File name to retrieve, defaults to
mafw-docs-<version>.ziptimeout_s (float) – Request timeout in seconds, defaults to 60.0
- Returns:
Path to the downloaded zip archive, or None if it does not exist or cannot be retrieved
- Return type:
Path | None
- mafw.devtools.documentation.ensure_sphinx_build_available() None
Ensure that the Sphinx Python package is available.
doc_versioningis a development helper shipped with MAFw. The script is typically executed from the optional[dev]environment (either by activating the development environment and using the console entry point, or viahatch run dev.py<version>:multidocon CI/CD).Checking for the
sphinx-buildexecutable alone is not sufficient because the effective availability depends on which Python environment is executing the command. Checking the import spec forsphinxvalidates that the correct optional dependencies are installed for the running interpreter.- Raises:
DevtoolsError – If Sphinx is not available.
- mafw.devtools.documentation.ensure_versions_json_exists(outdir: Path) bool
Ensure versions.json exists in outdir. If not, try to copy from another version.
- Parameters:
outdir (Path) – Output directory that should contain versions.json
- Returns:
True if versions.json exists or was successfully copied
- Return type:
bool
- mafw.devtools.documentation.extract_docs_zip_to_repo_root(zip_path: Path, repo_root: Path | None = None) None
Extract a documentation zip archive into the repository root and remove the zip.
The zip archive is expected to contain paths rooted at the repository (e.g.
docs/build/doc/vX.Y.Z/...) so that extraction recreates the same structure as a normal documentation build.- Parameters:
zip_path (Path) – Zip archive to extract
repo_root (Path | None) – Repository root directory, defaults to auto-detection
- Raises:
FileNotFoundError – If
zip_pathdoes not existzipfile.BadZipFile – If the archive is invalid
- mafw.devtools.documentation.filter_latest_micro(versions: List[Tuple[Version, Any]]) List[Tuple[Version, Any]]
Keep only the latest micro version per minor (major.minor).
- Parameters:
versions (List[Tuple[Version, Any]]) – List of (Version, tag) tuples
- Returns:
Filtered list of (Version, tag) tuples
- Return type:
List[Tuple[Version, Any]]
- mafw.devtools.documentation.filter_stable_tags(tags: List[str], regex: str) List[str]
Filter tags based on a regular expression pattern.
- Parameters:
tags (List[str]) – List of tag strings to filter
regex (str) – Regular expression pattern to match against
- Returns:
Filtered list of matching tags
- Return type:
List[str]
- mafw.devtools.documentation.filter_versions_in_range(versions: List[str], from_v: str | None, to_v: str | None) List[str]
Filter versions within an inclusive semantic-version range.
- Parameters:
versions (List[str]) – Input versions list
from_v (str | None) – Range start (inclusive)
to_v (str | None) – Range end (inclusive)
- Returns:
Filtered versions list
- Return type:
List[str]
- Raises:
ValueError – If range bounds are invalid
- mafw.devtools.documentation.find_repo_root(start: Path | None = None) Path
Find the repository root directory.
The root is detected by walking upwards until a
pyproject.tomlfile is found. If no such file is found, the starting directory is returned.- Parameters:
start (Path | None) – Directory from which to start searching, defaults to current working directory
- Returns:
Resolved repository root directory
- Return type:
Path
- mafw.devtools.documentation.format_size(bytes_size: float) str
Format bytes to human-readable size.
- Parameters:
bytes_size (int) – Size in bytes
- Returns:
Formatted size string
- Return type:
str
- mafw.devtools.documentation.format_started_at_utc(now: datetime) str
Format a timestamp for server metadata.
- Parameters:
now (datetime) – A datetime instance (timezone-aware is preferred)
- Returns:
UTC ISO-8601 timestamp with
Zsuffix- Return type:
str
- mafw.devtools.documentation.generate_pdf_index_page(html_outdir: Path, pdf_info: List[dict[str, str]], project_name: str = 'Documentation') None
Generate an HTML page listing all available PDFs. This page will be placed in the root html_versions directory. Order: stable first, then latest, then other releases sorted by version (newest first).
- Parameters:
html_outdir (Path) – Output directory for HTML files
pdf_info (List[dict[str, str]]) – List of dictionaries containing PDF information
project_name (str) – Name of the project for the page title, defaults to ‘Documentation’
- mafw.devtools.documentation.generate_python_versions_rst(*, repo_root: Path | None = None) None
Generate the RST substitution file for the supported Python range.
The file is emitted under
docs/source/requirementsso it can be included by the general documentation and copied into the README update block.- Parameters:
repo_root (Path | None) – Repository root directory, defaults to auto-detection
- mafw.devtools.documentation.generate_requirements_rst(group_name: str = 'base', *, repo_root: Path | None = None) None
Generate an RST file with the dependencies of a given group.
The function parses pyproject.toml to retrieve the dependencies and their descriptions from the tool.mafw.dependency-description section. The generated file is saved as <group_name>_requirements.rst in the docs/source directory.
- Parameters:
group_name (str) – The name of the dependency group (e.g., ‘base’, ‘seaborn’), defaults to ‘base’
repo_root (Path | None) – Repository root directory, defaults to auto-detection
- mafw.devtools.documentation.get_current_branch() str
Get the name of the currently checked out branch.
- Returns:
Name of the current branch
- Return type:
str
- mafw.devtools.documentation.get_directory_size(path: Path) int
Calculate total size of a directory in bytes.
- Parameters:
path (Path) – Directory path
- Returns:
Total size in bytes
- Return type:
int
- mafw.devtools.documentation.get_git_tags(min_version: str | None = None) List[Tuple[Version, Any]]
Return list of (Version, tag) tuples sorted ascending.
- Parameters:
min_version (str | None) – Minimum version to consider, defaults to None
- Returns:
List of (Version, tag) tuples
- Return type:
List[Tuple[Version, Any]]
- mafw.devtools.documentation.git_rev_of(ref: str) str
Get the git revision hash for a given reference.
- Parameters:
ref (str) – Git reference (tag, branch, commit hash)
- Returns:
Git revision hash
- Return type:
str
- Raises:
RuntimeError – If git rev-list fails
- mafw.devtools.documentation.http_probe(address: str, port: int, timeout_s: float = 1.0) bool
Probe whether the server responds at the given address:port.
- Parameters:
address (str) – Server address
port (int) – Server port
timeout_s (float) – Timeout in seconds
- Returns:
True if a GET to the server root returns HTTP 200
- Return type:
bool
- mafw.devtools.documentation.is_ancestor(a: str, b: str) bool
Return True if commit a is ancestor of commit b (git merge-base –is-ancestor).
- Parameters:
a (str) – First commit reference
b (str) – Second commit reference
- Returns:
True if a is ancestor of b
- Return type:
bool
- mafw.devtools.documentation.iter_local_mafw_docs_zips(zip_dir: Path) List[Tuple[str, Path]]
List local mafw-docs zip files in a directory.
Only files matching
mafw-docs-vX.Y.Z.zipare returned.- Parameters:
zip_dir (Path) – Directory to scan
- Returns:
List of (version, file_path) tuples
- Return type:
List[Tuple[str, Path]]
- mafw.devtools.documentation.list_mafw_docs_generic_packages(api_config: GitlabAPIConfiguration, package_name: str = 'mafw-docs') List[dict[str, Any]]
List generic packages for mafw-docs in the GitLab Package Registry.
Uses the Packages API:
GET /projects/:id/packagesand filters forpackage_type=genericand exactname == package_name.- Parameters:
api_config (GitlabAPIConfiguration) – GitLab API configuration
package_name (str) – Package name, defaults to
mafw-docs
- Returns:
List of package dictionaries from the API
- Return type:
List[dict[str, Any]]
- mafw.devtools.documentation.mirror_version(outdir: Path, src_tag: str, target_tag: str, use_symlink: bool = True) None
Mirror a version directory from one tag to another. Can use symlinks for efficiency or copy for compatibility.
- Parameters:
outdir (Path) – Output directory containing version directories
src_tag (str) – Source tag directory name
target_tag (str) – Target tag directory name
use_symlink (bool) – Whether to use symlink instead of copying, defaults to True
- mafw.devtools.documentation.normalize_registry_item(item: str) Tuple[str, str]
Normalize a registry item into (version, file_name).
The item can be either: - a version string:
vX.Y.Z- a file name:mafw-docs-vX.Y.Z.zip- Parameters:
item (str) – Input item
- Returns:
Tuple of (version, file_name)
- Return type:
Tuple[str, str]
- Raises:
ValueError – If the item cannot be normalized
- mafw.devtools.documentation.parse_mafw_docs_zip_filename(file_name: str) Tuple[str, str] | None
Parse and validate a mafw-docs zip filename.
The accepted filename pattern is:
mafw-docs-vX.Y.Z.zip.- Parameters:
file_name (str) – File name to parse
- Returns:
Tuple of (version, normalized_file_name) if valid, otherwise None
- Return type:
Tuple[str, str] | None
- mafw.devtools.documentation.parse_sphinx_log(log_content: str) Tuple[int, int, List[str]]
Parse Sphinx build log to extract warning and error counts, and warning messages.
Only three warnings are reported
- Parameters:
log_content (str) – Sphinx build log
- Returns:
Tuple of warning, error count, warning messages
- Return type:
Tuple[int, int, List[str]]
- mafw.devtools.documentation.parse_version_tuple(tag: str) Tuple[int, ...]
Parse vX.Y.Z(.W) into tuple of ints for sorting.
- Parameters:
tag (str) – Version tag string
- Returns:
Tuple of integers representing the version
- Return type:
Tuple[int, …]
- mafw.devtools.documentation.process_matches_http_server(proc: Any, address: str, port: int, directory: Path) bool
Check whether a process resembles the expected http.server invocation.
- Parameters:
proc (Any) – psutil Process instance
address (str) – Expected bind address
port (int) – Expected port
directory (Path) – Expected served directory
- Returns:
True when the process signature matches
- Return type:
bool
- mafw.devtools.documentation.prune_old_versions(outdir: Path, max_size_mb: int = 100, dry_run: bool = False) Tuple[List[str], int]
Remove oldest version directories until total size is below threshold. Always keeps ‘stable’, ‘latest’, and ‘dev’ (if present).
- Parameters:
outdir (Path) – Output directory containing version directories
max_size_mb (int) – Maximum size in megabytes
dry_run (bool) – If True, only report what would be deleted
- Returns:
Tuple of (list of removed versions, final size in bytes)
- Return type:
Tuple[List[str], int]
- mafw.devtools.documentation.read_server_metadata(directory: Path) dict[str, Any] | None
Read local documentation server metadata if available.
- Parameters:
directory (Path) – Served directory to locate metadata under
- Returns:
Parsed metadata, or None when the metadata file is missing
- Return type:
dict[str, Any] | None
- Raises:
DevtoolsError – If the metadata file exists but cannot be parsed
- mafw.devtools.documentation.regenerate_versions_json_after_pruning(outdir: Path, removed_versions: List[str]) None
Regenerate versions.json after pruning, excluding removed versions.
- Parameters:
outdir (Path) – Output directory containing version directories
removed_versions (List[str]) – List of version names that were removed
- mafw.devtools.documentation.report_build_status(tag: str, success: bool, log: str, build_type: str = 'HTML') None
Report build status with warning/error summary.
- Parameters:
tag (str) – Version tag being built
success (bool) – Whether build succeeded
log (str) – Build log content
build_type (str) – Type of build (HTML or PDF)
- mafw.devtools.documentation.resolve_mafw_docs_package_ids_by_version(api_config: GitlabAPIConfiguration, package_name: str = 'mafw-docs') dict[str, int]
Resolve package IDs for mafw-docs generic packages by version.
- Parameters:
api_config (GitlabAPIConfiguration) – GitLab API configuration
package_name (str) – Package name, defaults to
mafw-docs
- Returns:
Mapping from version string to package id
- Return type:
dict[str, int]
- mafw.devtools.documentation.run(cmd: List[str], cwd: Path | None = None) CompletedProcess[str]
Helper to run commands with consistent behavior.
- Parameters:
cmd (List[str]) – Command to execute as a list of strings
cwd (Path | None) – Working directory for command execution, defaults to None
- Returns:
Completed process result
- Return type:
subprocess.CompletedProcess[str]
- mafw.devtools.documentation.server_default_directory() Path
Return the default directory served by the local documentation server.
- Returns:
Default documentation build directory
- Return type:
Path
- mafw.devtools.documentation.server_metadata_path(directory: Path) Path
Return the metadata path associated to a served directory.
- Parameters:
directory (Path) – Served directory
- Returns:
Metadata JSON file path
- Return type:
Path
- mafw.devtools.documentation.sort_tags_semver(tags: List[str]) List[str]
Sort tags using semantic versioning comparison.
- Parameters:
tags (List[str]) – List of tag strings to sort
- Returns:
Sorted list of tag strings
- Return type:
List[str]
- mafw.devtools.documentation.upload_docs_zip_to_gitlab_generic_registry(api_config: GitlabAPIConfiguration, package_version: str, zip_path: Path, package_name: str = 'mafw-docs', timeout_s: float = 60.0) bool
Upload a documentation zip archive to the GitLab Generic Package Registry.
The equivalent GitLab API endpoint is:
PUT /projects/:id/packages/generic/:package_name/:package_version/:file_nameAuthentication headers: -
JOB-TOKENwhen running on CI (api_config.on_ciis True) -PRIVATE-TOKENfor local execution- Parameters:
api_config (GitlabAPIConfiguration) – GitLab API configuration
package_version (str) – Package version (typically the git tag, e.g.
v2.1.0)zip_path (Path) – Path to the zip file to upload
package_name (str) – Generic package name, defaults to
mafw-docstimeout_s (float) – Request timeout in seconds, defaults to 60.0
- Returns:
True if the file was uploaded, False if the upload was skipped because the target already exists
- Return type:
bool
- Raises:
FileNotFoundError – If
zip_pathdoes not existRuntimeError – If the upload fails (non-2xx response)
- mafw.devtools.documentation.write_legacy_redirect_page(outdir: Path) None
Create a legacy redirect page at the root of the output directory.
- Parameters:
outdir (Path) – Output directory for the redirect page
- mafw.devtools.documentation.write_redirect_page(outdir: Path, name: str, target_tag: str) None
Create a redirect page for a version alias.
- Parameters:
outdir (Path) – Output directory for the redirect page
name (str) – Name of the redirect alias (e.g., ‘stable’, ‘dev’)
target_tag (str) – Tag that the redirect should point to
- mafw.devtools.documentation.write_redirects_file(outdir: Path) None
Create a _redirects file for GitLab Pages.
- Parameters:
outdir (Path) – Output directory for the redirects file
- mafw.devtools.documentation.write_root_landing_page(build_root: Path, project_name: str = 'MAFw') None
Create a landing page for the project root with links to documentation and coverage.
- Parameters:
build_root (Path) – Root build directory (should contain ‘doc’ subdirectory)
project_name (str) – Project name for the page title
- mafw.devtools.documentation.write_server_metadata(directory: Path, metadata: dict[str, Any]) None
Write local documentation server metadata.
- Parameters:
directory (Path) – Served directory where metadata is stored
metadata (dict[str, Any]) – Metadata dictionary to serialize as JSON
- Raises:
DevtoolsError – If the metadata file cannot be written
- mafw.devtools.documentation.write_versions_json(outdir: Path, versions: List[dict[str, str]]) None
Write versions information to a JSON file.
- Parameters:
outdir (Path) – Output directory for the JSON file
versions (List[dict[str, str]]) – List of version information dictionaries
Modules
Sphinx documentation building helpers for MAFw versioned documentation. |
|
Requirements documentation constants and generators for MAFw. |
|
Local documentation server management for MAFw. |
|
Version management helpers for MAFw versioned documentation. |