Using validators

trudag can be used to validate any data that can be reduced to a floating point metric. However, trudag does not provide any validation functionality out of the box: we expect users to have their own use cases that are highly specific to their Artifacts. This page describes how to write and integrate your validators.

A Validator is any type-hinted function with the signature

<Signature (configuration: dict[str, yaml]) -> tuple[float, list[Exception | Warning]]>

where yaml is an alias defined as

yaml: TypeAlias = str | int | float | bool | list["yaml"] | dict[str, "yaml"]

Writing a validator

As an example, suppose we would like to compare the response time of an http request to some target. First, write the function signature. Remember, the function name will be used to identify the validator in item files, so choose something sensible.

def https_response_time(configuration: dict[str, yaml]) -> tuple[float, list[Exception | Warning]]:

Next, decide what the entry in a Markdown file will look like for your validator. For our use case, the following yaml fields seem sensible:

evidence:
    type: https_response_time
    configuration:
        target_microseconds: # Target to be met, or beaten.
        url: # url to query

The contents of the configuration field will be passed directly to the validator as a dict.

We can now write the body of our validator. A good validator will always return a score; remember it is always safe to be conservative and return 0.0.

def https_response_time(configuration: dict[str, yaml]) -> tuple[float, list[Exception | Warning]]:
    target = configuration.get("target_microseconds", None)
    url = configuration.get("url", None)
    if not url:
        return (0.0, [ValueError("No url specified for https_response_time validator")])
    if not target_microseconds:
        return (0.0, [ValueError("No target time specified for https_response_time validator")])
    response = requests.get("https://" + url)
    score = min(target/response.elapsed.microseconds, 1.0)
    return (score, [])

Tip

Validators are used by default when scoring items, you can pass --no-validate to skip using validators.

Check the usage for more information.

Adding as a local extension

To add https_response_time as a local extension, create a new file.dotstop_extensions/validators.py in the current working directory. All functions with the signature <Signature (configuration: dict[str, yaml]) -> tuple[float, list[Exception | Warning]]> available in this namespace will now be usable within trudag, accessible using their method name.

Packaging as a plugin

To mark a package as containing validator plugins, add the following entry point to its pyproject.toml:

[project.entry-points."trustable.validator.plugins"]
<ref> = <object>

All functions with the signature <Signature (configuration: dict[str, yaml]) -> tuple[float, list[Exception | Warning]]> available in this namespace will now be usable within trudag, again accessible using their method name.