Uploading packages

Workflow package_upload

This workflow signs and uploads source and/or binary packages to an upload queue. It is normally expected to be used as a sub-workflow.

  • task_data:

    • source_artifact (Single lookup, optional): a debian:source-package or debian:upload artifact representing the source package (the former is used when the workflow is started based on a .dsc rather than a .changes)

    • binary_artifacts (Multiple lookup, optional): a list of debian:upload artifacts representing the binary packages

    • merge_uploads (boolean, defaults to False): if True, merge the uploads and create a single PackageUpload task to upload them all together; if False, create a separate PackageUpload task for each upload

    • since_version (string, optional): passed to MakeSourcePackageUpload task if source_artifact is a debian:source-package

    • target_distribution (string, optional): passed to MakeSourcePackageUpload task if source_artifact is a debian:source-package

    • key (Single lookup, optional): the debusine:signing-key artifact to sign the upload with, which must have purpose openpgp

    • require_signature (boolean, defaults to True): whether the upload must be signed

    • target (required): the upload queue, as an ftp:// or sftp:// URL

    • vendor (string, optional): the distribution vendor to use for running MakeSourcePackageUpload task and MergeUploads task

    • codename (string, optional): the distribution codename to use for running MakeSourcePackageUpload task and MergeUploads task

At least one of source_artifact and binary_artifacts must be set.

The workflow creates the following tasks, each of which has a dependency on the previous one in sequence, using event reactions to store output in the workflow’s internal collection for use by later tasks:

  • if source_artifact is a debian:source-package artifact: a MakeSourcePackageUpload task (with since_version and target_distribution) to build a corresponding .changes file Uses vendor and codename to construct the environment lookup.

  • if merge_uploads is True and there is more than one source and/or binary artifact: a MergeUploads task to combine them into a single upload. Uses vendor and codename to construct the environment lookup.

  • for each upload (or for the single merged upload, if merging):

    • if key is provided: a Debsign task to have debusine sign the upload with the given key

    • if key is not provided and require_signature is True: an ExternalDebsign task to wait until a user provides a signature, which debusine will then include with the upload

    • a PackageUpload task, to upload the result to the given upload queue

UI changes

Wait tasks will generally require some form of UI change. In particular, when showing a blocked ExternalDebsign work request, the web UI advises the user to run a new client command (e.g. debusine provide-signature <work request ID>, which downloads the unsigned upload, signs it, and uploads it back to the external-debsign API view above.

Pipeline considerations

The package_upload workflow will typically be used as a sub-workflow of a smart “pipeline” workflow. There are three main use cases:

  • Upload a source package to debusine, have it tested, and then have debusine pass on that upload to an external upload queue.

    In this case, the package_upload workflow’s task data should set source_artifact to the source package and leave binary_artifacts empty.

  • Upload a source package to debusine, have it tested, and then have debusine upload both the source and all built binaries to an external upload queue. (For example, this is useful when uploading a package to Debian that will land in the NEW queue, since Debian currently requires binaries for NEW uploads.)

    In this case, the package_upload workflow’s task data should set source_artifact to the source package, set binary_artifacts to a list of single lookups matching each of the binary uploads from the super-workflow’s internal collection (e.g. [internal@collections/name:build-all, internal@collections/name:build-amd64], and set merge_uploads to True.

  • debusine acts as a build daemon, building a source package for a number of architectures and uploading each of them as soon as the builds finish.

    In this case, the package_upload workflow’s task data should leave source_artifact unset, set binary_artifacts to a list of single lookups matching each of the binary uploads from the super-workflow’s internal collection, and set merge_uploads to False.

Generic code for creating child work requests using artifacts from the workflow’s internal collection adds appropriate dependencies on the work requests that are expected to provide those artifacts.

If the parent workflow needs some kind of manual validation step to complete before starting the upload (typically in the case of manual uploads but not when acting as a build daemon), it should add a dependency from the package_upload sub-workflow to the validation workflow step. The package_upload sub-workflow will be populated before validation is complete (since the root workflow handles population of all its sub-workflows), but it will not start running until validation is complete.