Remote Graph

TSF graphs are easier to manage when broken down into smaller, verifiable projects that can be integrated into larger graphs. A Remote Graph is one of these composable building blocks. It is a finished artifact or snapshot of a TSF graph that you can depend on but not change.

Currently, remote graphs are implemented as artifacts: files that package all the data required from one graph for reference by a different graph that builds upon it.

What is a Remote Graph?

Think of a remote graph like a published library:

• It is immutable: once published, it never changes.
• You must use it as-is: you cannot edit its internals.
• It defines clear needs: it tells you what you must resolve in order for you to use it safely.
• It acts as an interface: connecting what the remote graph provides with what your local graph must supply.

The most common use-case of remote graphs is the integration of upstream project's graph into downstream project's graph. Upstream can provide argumentation and declare needs which downstream can reference and substantiate respectively. This enables traceability between the two projects, without either requiring access to the other's working graph.

Parts of a Remote Graph

A remote graph usually contains two complementary pieces: the Resolved Graph and the Needs Graph.

Resolved Graph

The resolved graph is a frozen snapshot of a graph at a specific point in time, everything that has already been computed (Validators) or processed (References).

Key points:

• Includes pre-computed scores and references
• Read-only: you cannot attach new local items directly into it.
• Exports content and references used to build it.
  • Including needs from another remote graph.
    • Does not expose transitive dependencies (you can’t automatically see what it depends on).

Needs Graph

The needs graph describes what is not yet resolved. This is what your local graph must provide in order to make use of the remote graph. Think of it as the “contract clauses” of the artifact.

Key points:

• Includes unresolved assumptions such as AoUs (Assumptions of Use).
• Imported into your local graph under a namespace for clarity.
• Must be scored locally, even if you choose to ignore parts of it.
  • Important: ignored assumptions must still be documented.
• Ensures your local graph complies with the requirements of the remote graph.

Usage

Consumed artifacts should be associated with the same revision of the project that is being used. If you are using v0.3.0 of a project, the artifact consumed should also have been generated at release v0.3.0. It is therefore recommended that producers of artifacts export on release as part of a CI pipeline, such that the artifact is versioned, and is the source of truth.

Needs

Needs document dependencies (or 'assumptions of use') that are defined by a graph that need to be satisfied in the context of a downstream consumer of that graph. These typically request evidence and/or assertions that cannot be defined in the originating graph, because they are context-specific. They may also document known limitations or constraints on the usage of the associated software, or known gaps in the reasoning or evidence provided.

Initialising and Populating the Needs Graph

To create and operate on the "needs" graph, use the top-level --needs option:

• trudag --needs init will create the .needs.dot file.
• trudag --needs manage create-item ... will create an item in the needs graph.
• trudag --needs manage move-item --to-dotstop MY-ITEM will move MY-ITEM to dotstop, from needs.

Any trudag command, except export, can be run with this option; the needs graph is managed in the same way the main graph is managed.

Moving existing items into the needs graph

If an item already exists in the dotstop graph and should be treated as a "need" it can be moved using move-item:

trudag manage move-item --to-needs MY-ITEM

The item must have no links (parents or children) in the source graph before it can be moved. The underlying markdown file is not moved or deleted, only the graph entry is transferred between .dotstop.dot and .needs.dot.

To move an item back from needs to dotstop:

trudag --needs manage move-item --to-dotstop MY-ITEM

Guidance on Writing Needs

Needs are written in the same manner as assertions, and still provide argumentation about the given project, but differ in that it is up to the consumer of the artifact to provide evidence that the needs are fulfilled. Needs can contain references, but SME scores and validators are not permitted. For example:

---
references:
  - type: file
    path: path_to_reference.txt
---

Project XYZ is built reproducibly with using tooling ABC.

Producing an Artifact

Run trudag export --artifact <path> --project-name <name> to write the artifact to a file specified by --artifact. This will both resolve references and run validators for the graph, so any required plugins must be available when performing an export.

This command will fail if any references fail, as issues can propagate to artifact consumers. However, the option --allow-failure is available if you wish to ignore this.

Warning

Though it is not required, it is highly recommended that all items and links are reviewed before exporting, particularly if the artifact is for a release.

Consuming an Artifact

Currently, when working with artifacts, we expect the artifact to be present as a file in a reachable directory.

Importing Needs

For a consumer project, once the artifact has been obtained, run trudag import to import the needs items into the local trustable graph. This command uses the following parameters:

• --artifact: (required) The path from which to read the artifact
• --import-dir/-d: (required) The path to the directory where imported items should be extracted.
• --namespace/-n: (required) Prefix for any extracted items. This helps resolve any name conflicts.

During the import process, the needs items will be created in the specified directory, prefixed with the namespace, and imported into the consumer project's graph.

The consumer project can now use these needs items as any other item in their local project (including attaching evidences for them).

Referencing the Resolved Graph

The ArtifactReference allows an item in the local graph to reference a subgraph of an artifact's resolved graph. For example, an item with the following frontmatter will reference the descendant subgraph with roots ITEM-1 and ITEM-2:

references:
  - type: artifact
    path: path_to_artifact.json
    roots:
      - ITEM-1
      - ITEM-2

Consequently:

• Any changes to the referenced items will prompt a review for the referencing item.
• This may be the case if a newer artifact is pulled.
• The published report will also publish a report of the referenced artifacts. Referenced items can be clicked through to in the rendered reference.

Planned Features

• Version migration support: smoother upgrades between remote graph versions.
• Backward compatibility:
  • Load newer versions with warnings if they break rules.
  • Safely use older versions without issues.
• Explicit exports: if something should be shared, it must be marked public (similar to public vs. private in OOP).
• Implementation for Read Only enforcement of Resolved Graph.