# The debputy manifest format

_This is [reference guide] and is primarily useful if you have an idea of what you are looking for._
_If you are new to `debputy`, maybe you want to read [GETTING-STARTED-WITH-dh-debputy.md](GETTING-STARTED-WITH-dh-debputy.md) first._

<!-- To writers and reviewers: Check the documentation against https://documentation.divio.com/ -->


## Prerequisites

This guide assumes familiarity with Debian packaging in general.  Notably, you should understand
the different between a (Debian) source package and a (Debian) binary package (e.g., `.deb`) plus
how these concepts relates to `debian/control` (the source control file).

Additionally, the reader is expected to have an understanding of globs and substitution variables.

It is probably very useful to have an understanding on how a binary package is
assembled.  While `debputy` handles the assembly for you, this document will not go in details
with this. Prior experience with `debhelper` (notably the `dh`-style `debian/rules`) may be
useful but should not be a strict requirement.


# The basic manifest

The manifest is a YAML document with a dictionary at the top level layer.  You can choose to leave
for YAML version be implicit.  All manifests must include a manifest version, which will enable
the format to change over time.  For now, there is only one version (`0.1`) and you have to include
the line:

    manifest-version: "0.1"


On its own, the manifest containing only `manifest-version: "..."` will not do anything.  So if you
end up only having the `manifest-version` key in the manifest, you can just remove the manifest and
rely entirely on the built-in rules.


# Path matching rules

Most of the manifest is about declaring rules for a given path such as "foo must be a symlink"
or "bar must be owned by root:tty and have mode 02755".  

The manifest contains the following types of matches:

 1) Exact path matches.  These specify the path inside the debian package exactly without any
    form of wildcards (e.g., `*` or `?`).  However, they can use substitution variables.
    Examples include:
    * `usr/bin/sudo`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/libfoo.so`

    Having a leading `/` is optional.  I.e. `/usr/bin/sudo` and `usr/bin/sudo` are considered the
    same path.

 2) Glob based path matches.  These specify a rule that match any path that matches a given
    glob.  These rules must contain a glob character (e.g., `*`) _and_ a `/`. Examples include:

    * `foo/*`
    * `foo/*.txt`
    * `usr/lib/{{DEB_HOST_MULTIARCH}}/lib*.so*`

    Note that if the glob does not contain a `/`, then it falls into the Basename glob rule
    below.

 3) Basename glob matches.  These specify a rule that match any path where the basename matches
    a given glob.  They must contain a glob character (e.g., `*`) but either must not have a
    `/` in them at all or start with `**/` and not have any `/` from there.
    Examples include:

    * `"*.la"`
    * `"**/*.md"`

    The examples use explicit quoting because YAML often interprets `*` as an anchor rule in the
    places where you are likely to use this kind of glob.  The use of the `**/`-prefix is
    optional. 

However, there are also cases where these matching rules can cause conflicts.  This is covered
in the [Conflict resolution](#Conflict resolution) section below.


## Limitations on debputy path rules

Path rules:

 1) Must match relatively to the package root.
 2) Never resolves a symlink.
    * If `bin/ls` is a symlink, then `bin/ls` matches the symlink (and never the target)
    * If `bin` is a symlink, then `bin/ls` (or `bin/*`) would fail to match, because
      matching would require resolving the symlink.

These limitations are in place because of implementation details in `debputy`.


## Conflict resolution

The manifest can contain seemly mutually exclusive rules.  As an example, if you ask for
`foo/symlink` to be a symlink but also state that you want to remove `foo` entirely
from the debian package then the manifest now has two mutually exclusive requests.

To resolve these problems, `debputy` relies on the following rules for conflict resolutions:

  1. Requests are loosely-speaking ranked and applied from top to bottom:
     1. `installations` (ordered; "first match wins")
     2. `transformations` (ordered; "all matching rules applies")

     The "first match wins" rule is exactly what it says.

     The "all matching rules applies" rule means that each rule is applied in order.  Often
     this behaves like "last match wins", but if a transformation modifies the file system
     layout, then it affects later matches.

  2. The ordered rules (such as every transformation inside `transformations`) are applied in the
     listed order (top to bottom). Due to the explicit ordering, such rules generally do not trigger
     other conflict resolution rules.

     Note keep in mind that all rules must in general apply to at least one path.

Note that `debputy` will in some cases enforce that rules are not redundant.  This feature is currently
only fully implemented for `installations`.

## All definitions must be used

Whenever you define a request or rule in the manifest, `debputy` will insist on it being used
at least once.  The exception to this rule being conditional rules where the condition
evaluates to `false` (in which case the rule never used and does not trigger an error).

This is useful for several reasons:

 1. The definition may have been shadowed by another rule and would have been ignored otherwise
 2. The definition may no longer be useful, but its present might confuse a future reader of
    the manifest.

In all cases, `debputy` will tell you if a definition was unused and where you can find that
definition.


## debputy globs

In general, the following rules applies to globs in `debputy`.

 * The `*` match 0 or more times any characters except `/`.
 * The `?` match exactly one character except `/`.
 * The glob `foo/*` matches _everything_ inside `foo/` including hidden files (i.e., paths starting 
   with `.`) unlike `bash`/`sh` globs. However, `foo/*` does _not_ match `foo/` itself (this latter
   part matches `bash`/`sh` behaviour and should be unsurprising).
 * For the special-cases where `**` is supported, then `**` matches zero or more levels of directories.
   This means that `foo/**/*` match any path beneath `foo` (but still not `foo`).  This is mostly relevant
   for built-in path matches as it is currently not possible to define `foo/**/...` patterns in the manifest.


# Substitution rules

The `debputy` tool supports substitution in various places (usually paths) via the following
rules.  That means:

 1) All substitutions must start with `{{` and end with `}}`.  The part inbetween is
    the `VARIABLE` and must match the regular expression `[A-Za-z0-9][-_:0-9A-Za-z]*`.
    Note that you can use space around the variable name if you feel that increases readability.
    (That is, `{{ FOO }}` can be used as an alternative to `{{FOO}}`).
 2) The `VARIABLE` will be result from a set of builtin variables and the variables from
    `dpkg-architecture`.
 3) You can use `{{DOUBLE_OPEN_CURLY_BRACE}}` and `{{DOUBLE_CLOSE_CURLY_BRACE}}` (built-in variables)
    if you want a literal `{{` or `}}`  would otherwise have triggered an undesired expansion.
 4) All `{{VARIABLE}}` must refer to a defined variable.
 5) There are no expression syntax inside the `{{ ... }}` (unlike jinja2 and other template languages).
    This rule may be changed in the future (with a new manifest version).

Keep in mind that substitution _cannot_ be used everywhere.  There are specific places where
it can be used.  Also, substitution _cannot be used_ to introduce globs into paths.  When a
substitution occurs inside a path all characters inserted are treated as literal characters.


## Builtin or common substitution variables

 * `{{NEWLINE}}` or `{{NL}}` expands to a literal newline (LF) `\n`.
 * `{{TAB}}` expands to a literal tab `\t`.
 * `{{OPEN_CURLY_BRACE}}`  / `{{CLOSE_CURLY_BRACE}}` expand to `{` / `}`
 * `{{DOUBLE_OPEN_CURLY_BRACE}}` / `{{DOUBLE_CLOSE_CURLY_BRACE}}` expands to `{{` / `}}`.
 * `{{PACKAGE}}` expands to the binary package name of the current binary package. This substitution
   only works/applies when the substitution occurs in the context of a concrete binary package.
 * Plus any of the variables from produced by `dpkg-architecture`, such as `{{DEB_HOST_MULTIARCH}}`.

The following variables from `/usr/share/dpkg/pkg-info.mk` (`dpkg`) are also available:

 * DEB_SOURCE (as `{{DEB_SOURCE}}`)
 * DEB_VERSION (as `{{DEB_VERSION}}`)
 * DEB_VERSION_EPOCH_UPSTREAM (as `{{DEB_VERSION_EPOCH_UPSTREAM}}`)
 * DEB_VERSION_UPSTREAM_REVISION (as `{{DEB_VERSION_UPSTREAM_REVISION}}`)
 * DEB_VERSION_UPSTREAM (as `{{DEB_VERSION_UPSTREAM}}`)
 * SOURCE_DATE_EPOCH (as `{{SOURCE_DATE_EPOCH}}`)

These have the same definition as those from the `dpkg` provided makefile.


# Restrictions on defining ownership of paths

In some parts of the manifest, you can specify which user or group should have ownership of
a given path.  As an example, you can define a directory to be owned by `root` with group `tty`
(think `chown root:tty <some/path>`).

Ownership is generally defined via the keys `owner` and `group`.  For each of them, you can use
one of the following formats:

 1) A name (e.g., `owner: root`).
 2) An id (e.g., `owner: 0`). Please avoid using quotes around the ID in YAML as that can
    cause `debputy` to read the number as a name.
 3) A name and an id with a colon inbetween (e.g., `owner: "root:0"`).  The name must always
    come first here.  You may have to quote the value to prevent the YAML parser from being
    confused.

All three forms are valid and provide the same result.  Unless you have a compelling reason to
pick a particular form, the name-only is recommended for simplicity.  Notably, it does not
require your co-maintainer or future you to remember what the IDs mean.

Regardless of which form you pick:

 1) The provided owner must be defined by Debian `base-passwd` file, which are the only users guaranteed
    to be present on every Debian system.
    * Concretely, `debputy` verifies the input against `/usr/share/base-passwd/passwd.master` and
      `/usr/share/base-passwd/group.master` (except for `root` / `0` as an optimization).

 2) If the `name:id` form is used, then the name and the id values must match.  I.e., `root:2` would
    be invalid as the id for `root` is defined to be `0` in the `base-passwd` data files.

 3) The `debputy` tool maintains a `deny`-list of owners that it refuses even though `base-passwd`
    defines them. As a notable non-exhaustive example, `debputy` considers `nobody` or id `65534` 
    (the ID of `nobody` / `nogroup`) to be invalid owners.


# Conditional rules

There are cases, where a given rule should only apply in certain cases - such as only when a given
build profile is active (`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`).  For rules that
*support being conditional*, the condition is generally defined via the `when:` key and the condition
is then described beneath the `when:`.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      arch-matches: '!hurd-any'


When the condition under `when:` resolves to `true`, the rule will and must be used.  When the
condition resolves to `false`, the rule will not be applied even if it could have been.  However,
the rule may still be "partially" evaluated.  As an example, for installation rules, the source
patterns will still be evaluated to reserve what it would have matched, so that following rules
behave deterministically regardless of how the condition evaluates.

Note that conditions are *not* used as a conflict resolution and as such two conditional rules
can still cause conflicts even though their conditions are mutually exclusive.  This may be
changed in a later version of `debputy` provided `debputy` can assert the two conditions ensures
the rules are mutually exclusive.

The `when:` key has either a mapping, a list or a string as value depending on the condition.
Each supported condition is described in the following subsections.

## Package architecture match condition `arch-matches` (mapping)

The `arch-matches` condition is used to assert whether the content of the package is for a given
architecture.  The `arch-matches` condition cannot be applied to architecture independent packages
(`Architecture: all`) as the answer would not change between build and is therefore confusing and
redundant at best.

The `arch-matches` condition accounts for cross-compiling and will therefore match against
`DEB_HOST_ARCH` by default.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      arch-matches: '!hurd-any'

The `arch-matches` must be defined as a part of a mapping, where `arch-matches` is the key.  The
value is a string and will be the architecture restriction as a space separated list of architecture
names or architecture wildcards (same syntax as the architecture restriction in `Build-Depends`
in `debian/control` except there is no enclosing `[]` brackets). The names/wildcards can optionally
be prefixed by `!` to negate them.  However, either *all* names / wildcards must have negation or
*none* of them may have it.

## Active build profile match condition `build-profiles-matches` (mapping)

The `build-profiles-matches` condition is used to assert whether the active build profiles
(`DEB_BUILD_PROFILES` / `dpkg-buildpackage -P`) matches a given build profile restriction.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when:
                      build-profiles-matches: '<!pkg.foo.mycustomprofile>'

The `build-profiles-matches` must be defined as a part of a mapping, where `build-profiles-matches`
is the key.  The value is a string using the same syntax as the `Build-Profiles` field from `debian/control`
(i.e., a space separated list of `<[!]profile ...>` groups).

## Can run produced binaries `can-execute-compiled-binaries` (string)

The `can-execute-compiled-binaries` condition is used to assert the build can assume
that all compiled binaries can be run as-if they were native binaries. For native
builds, this condition always evaluates to `true`.  For cross builds, the condition
is generally evaluates to `false`.  However, there are special-cases where binaries
can be run during cross-building. Accordingly, this condition is subtly different
from the `cross-compiling` condition.

Note this condition should *not* be used when you know the binary has been built
for the build architecture (`DEB_BUILD_ARCH`) or for determining whether build-time tests
should be run (for build-time tests, please use the `run-build-time-tests` condition instead).
Some upstream build systems are advanced enough to distinguish building a final product vs.
building a helper tool that needs to run during build.  The latter will often be compiled by
a separate compiler (often using `$(CC_FOR_BUILD)`, `cc_for_build` or similar variable names
in upstream build systems for that compiler).

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  # Only for native builds or when we can transparently run a compiled
                  when: can-execute-compiled-binaries

The `can-execute-compiled-binaries` condition is specified as a string.

## Cross-Compiling condition `cross-compiling` (string)

The `cross-compiling` condition is used to determine if the current build is performing a cross
build (i.e., `DEB_BUILD_GNU_TYPE` != `DEB_HOST_GNU_TYPE`). Often this has consequences for what
is possible to do.

Note if you specifically want to know:

 * whether build-time tests should be run, then please use the `run-build-time-tests` condition.
 * whether compiled binaries can be run as if it was a native binary, please use the
   `can-execute-compiled-binaries` condition instead.  That condition accounts for cross-building
   in its evaluation.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when: cross-compiling

The `cross-compiling` condition is specified as a string.

## Whether build time tests should be run `run-build-time-tests` (string)

The `run-build-time-tests` condition is used to determine whether (build time) tests should
be run for this build.  This condition roughly translates into whether `nocheck` is present
in `DEB_BUILD_OPTIONS`.

In general, the manifest *should not* prevent build time tests from being run during cross-builds.

As an example:

    # TODO: Not the best example (`create-symlink` is an unlikely use-case for this condition)
    packages:
        foo:
            transformations:
            - create-symlink
                  path: sbin/agetty
                  target: /sbin/getty
                  when: run-build-time-tests

The `run-build-time-tests` condition is specified as a string.

## Negated condition `not` (mapping)

It is possible to negate a condition via the `not` condition.

As an example:

    packages:
        util-linux:
            transformations:
            - create-symlink
                  path: sbin/getty
                  target: /sbin/agetty
                  when:
                      # On Hurd, the package "hurd" ships "sbin/getty".
                      # This example is an alternative to `arch-marches: '!hurd-any`
                      not:
                          arch-matches: 'hurd-any'

The `not` condition is specified as a mapping, where the key is `not` and the
value is a nested condition.

## All or any of a list of conditions `all-of`/`any-of` (list)

It is possible to aggregate conditions using the `all-of` or `any-of` condition. This provide
`X and Y` and `X or Y` semantics (respectively).

As an example:

    packages:
        util-linux:
            transformations:
                - create-symlink
                  path: sbin/getty
                  target: /sbin/agetty
                  when:
                      # Only ship getty on linux except for s390(x)
                      all-of:
                          - arch-matches: 'linux-any'
                          - arch-matches: '!s390 !s390x'

The `all-of` and `any-of` conditions are specified as lists, where each entry is a nested condition.
The lists need at least 2 entries as with fewer entries the `all-of` and `any-of` conditions are
redundant.

# Installations

For source packages building a single binary, the `dh_auto_install` from debhelper will default to
providing everything from upstream's install in the binary package.  The `debputy` tool matches this
behaviour and accordingly, the `installations` feature is only relevant in this case when you need to
manually specify something upstream's install did not cover.

For sources, that build multiple binaries, where `dh_auto_install` does not detect anything to install,
or when `dh_auto_install --destdir debian/tmp` is used, the `installations` section of the manifest is
used to declare what goes into which binary package. An example:

    installations:
      - install:
            sources: "usr/bin/foo"
            into: foo
      - install:
            sources: "usr/*"
            into: foo-extra


All installation rules are processed in order (top to bottom).  Once a path has been matched, it can
no longer be matched by future rules.  In the above example, then `usr/bin/foo` would be in the `foo`
package while everything in `usr` *except* `usr/bin/foo` would be in `foo-extra`.  If these had been
ordered in reverse, the `usr/bin/foo` rule would not have matched anything and caused `debputy`
to reject the input as an error on that basis.  This behaviour is similar to "DEP-5" copyright files,
except the order is reversed ("DEP-5" uses "last match wins", where here we are doing "first match wins")

In the rare case that some path need to be installed into two packages at the same time, then this is
generally done by changing `into` into a list of packages.

All installations are currently run in *source* package context.  This implies that:

  1) No package specific substitutions are available. Notably `{{PACKAGE}}` cannot be resolved.
  2) All conditions are evaluated in source context.  For 99.9% of users, this makes no difference,
     but there is a cross-build feature that changes the "per package" architecture which is affected.

This is a limitation that should be fixed in `debputy`.

**Attention debhelper users**: Note the difference between `dh_install` (etc.) vs. `debputy` on
overlapping matches for installation.

**Limitations**: Note due to an implementation detail, the patterns in the `installation` section is
currently a bit more limited than elsewhere.

## Install rule search directories

Most install rules apply their patterns against search directories such as `debian/tmp` by default.

The default search directory order (highest priority first) is:

 1) The upstream install directory (usually, `debian/tmp`)
 2) The source package root directory (`.`)

Each search directory is tried in order.  When a pattern matches an entry in a search directory (even
if that entry is reserved by another package), further search directories will *not* be tried. As an example,
consider the pattern `usr/bin/foo*` and the files:

  `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh`
  `SOURCE_ROOT/usr/bin/foo.pl`

Here the pattern will only match `SOURCE_ROOT/debian/tmp/usr/bin/foo.sh` and not `SOURCE_ROOT/usr/bin/foo.pl`.

## Default discard rules

The `debputy` framework provides some built-in discard rules that are applied by default during installation
time.  These are always active and implicit, but can be overwritten by exact path matches for install rules.

The `debputy` tool itself provides the following discard rules:

 * Discard of `.la` files. Their use is rare but not unheard of. You may need to overwrite this.
 * Discard of python byte code (such as `__pycache__` directories).
 * Discard of editor backup files (such as `*~`, `*.bak`, etc.).
 * Discard of Version control files (such as `.gitignore`, etc.).
 * Discard of GNU info's `dir` (`usr/share/info/dir`) as it causes file conflicts with other packages.
 * Discard of `DEBIAN` directory.

If you find yourself needing a particular path installed that has been discarded by default, you can overrule
the default discard by spelling out the path. As an example, if you needed to install a `libfoo.la` file,
you could do:

    installations:
      - install:
            sources:
            # By-pass default discard of `libfoo.la` - note, no globs can be used!
             - "usr/lib/libfoo.la"
             - "usr/lib/libfoo*.so*"
            into: libfoo1

## Generic install (`install`)

The generic `install` rule can be used to install arbitrary paths into packages and is *similar* to how
`dh_install` from debhelper works.  It is a two "primary" uses.

  1) The classic "install into directory" similar to the standard `dh_install`
  2) The "install as" similar to `dh-exec`'s `=>` feature.

Examples:

    installations:
      - install:
            source: "usr/bin/tool-a"
            dest-dir: "usr/bin"
            into: foo
      - install:
            source: "usr/bin/tool-b"
            # Implicit dest-dir: "usr/bin
            into: foo-extra
      - install:
            source: "usr/bin/tool-c.sh"
            # Rename as a part of installing.
            as: "usr/bin/tool-c"
            into: foo-util


The value for `install` is a string, a list of strings or mapping. When it is a mapping, the
mapping has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
     matched, then the directory is installed along with all the contents that have not already been installed
     somewhere.

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed. This key is conditional on whether there are multiple binary packages listed in
   `debian/control`.  When there is only one binary package, then that binary is the default for `into`.
   Otherwise, the key is required.

 * `dest-dir` (optional): A path defining the destination *directory*.  The value *cannot* use globs, but can
   use substitution.  If neither `as` nor `dest-dir` is given, then `dest-dir` defaults to the directory name
   of the `source`.

 * `as` (optional): A path defining the path to install the source as. This is a full path.  This option is
   mutually exclusive with `dest-dir` and `sources` (but not `source`).  When `as` is given, then `source` must
   match exactly one "not yet matched" path.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules). 

When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.


## Install documentation and examples (`install-docs` and `install-examples`)

These install rules are related to documentation and resemble that of `dh_installdoc` / `dh_installexamples`
respectively.  They are shorthands over the generic `install` rule with the following key features:

 1) They pre-defined the `dest-dir` that respects the "main documentation package" recommendation from
    Debian Policy.
 2) They come with pre-defined conditional logic for skipping these rules under `DEB_BUILD_OPTIONS=nodoc`,
    so you do not have to write that conditional yourself.

With these two things in mind, they behave just like the `install` rule.

Examples:

    installations:
      - install-docs:
            sources:
              - "docs/README.some-topic.md"
              - "docs/README.another-topic.md"
            into: foo
      - install-examples:
            source: "examples/*"
            into: foo


The value for `install-docs` / `install-examples` is a string, a list of strings or mapping. When it is
a mapping, the mapping has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories.
    - When a symlink is matched, then the symlink (not its target) is installed as-is.  When a directory is
     matched, then the directory is installed along with all the contents that have not already been installed
     somewhere.

   * **CAVEAT**:  Specifying `source: examples` where `examples` resolves to a directory for `install-examples`
     will give you an `examples/examples` directory in the package, which is rarely what you want. Often, you
     can solve this by using `examples/*` instead. Similar for `install-docs` and a `doc` or `docs` directory.

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed as docs or examples.  This key is conditional on whether there are multiple binary packages
   listed in `debian/control`.  When there is only one binary package, then that binary is the default for
   `into`. Otherwise, the key is required.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.

Note: While the canonical name for these rules use plural, the `install-doc` and `install-example` variants
are accepted as alternative names.

## Install manpages (`install-man`)

Install rule for installing manpages similar to `dh_installman`. It is a shorthand over the generic
`install` rule with the following key features:

 1) The rule can only match files (notably, symlinks cannot be matched by this rule).
 2) The `dest-dir` is computed per source file based on the manpage's section and language.
 3) The rule comes with manpage specific attributes such as `language` and `section` for when the
    auto-detection is insufficient.
 4) The rule comes with pre-defined conditional logic for skipping the rule under `DEB_BUILD_OPTIONS=nodoc`,
    so you do not have to write that conditional yourself.

With these things in mind, the rule behaves similar to the `install` rule.

Examples:

    installations:
      - install-man:
            source: "man/foo.1"
            into: foo
      - install-man:
            source: "man/foo.de.1"
            language: derive-from-basename
            into: foo


The value for `install-man` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

 * `source` or `sources` (required): A path match (`source`) or a list of path matches (`sources`) defining
   the source path(s) to be installed. The path match(es) can use globs.  Each match is tried against default
   search directories. Only files can be matched.

 * `into` (conditional): Either a package name or a list of package names for which these paths should be
   installed as manpages.  This key is conditional on whether there are multiple binary packages
   listed in `debian/control`.  When there is only one binary package, then that binary is the default for
   `into`. Otherwise, the key is required.

 * `section` (optional): If provided, it must be an integer between 1 and 9 (both inclusive), defining the
    section the manpages belong overriding any auto-detection that `debputy` would have performed.

 * `language` (optional): If provided, it must be either a 2 letter language code (such as `de`), a 5 letter
   language + dialect code (such as `pt_BR`), or one of the special keywords `C`, `derive-from-path`, or
   `derive-from-basename`.  The default is `derive-from-path`.
   - When `language` is `C`, then the manpages are assumed to be "untranslated".
   - When `language` is a language code (with or without dialect), then all manpages matched will be assumed
     to be translated to that concrete language / dialect.
   - When `language` is `derive-from-path`, then `debputy` attempts to derive the language from the path
     (`man/<language>/man<section>`).  This matches the default of `dh_installman`. When no language can
     be found for a given source, `debputy` behaves like language was `C`.
   - When `language` is `derive-from-basename`, then `debputy` attempts to derive the language from the
     basename (`foo.<language>.1`) similar to `dh_installman` previous default.  When no language can
     be found for a given source, `debputy` behaves like language was `C`.  Note this is prone to
     false positives where `.pl`, `.so` or similar two-letter extensions gets mistaken for a language
     code (`.pl` can both be "Polish" or "Perl Script", `.so` can both be "Somali" and "Shared Object"
     documentation).  In this configuration, such extensions are always assumed to be a language.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).


When the input is a string or a list of string, then that value is used as shorthand for `source`
or `sources` (respectively).  This form can only be used when `into` is not required.

Comparison with debhelper: The `dh_installman` uses `derive-from-path` as default and then falls back
to `derive-from-basename`.  The `debputy` tool does *not* feature the same fallback logic.  If you want
the `derive-from-basename` with all of its false-positives, you have to explicitly request it.


## Discard (or exclude) upstream provided paths (`discard`)

When installing paths from `debian/tmp` into packages, it might be useful to ignore some paths that you never
need installed.  This can be done with the `discard` rule.


The value for `discard` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

 * `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
   source path(s) that should not be installed anywhere. The path match(es) can use globs.
    - When a symlink is matched, then the symlink (not its target) is discarded as-is.  When a directory is
     matched, then the directory is discarded along with all the contents that have not already been installed
     somewhere.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules).  This condition will
   be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Once a path is discarded, it cannot be matched by any other install rules.  A path that is discarded, is
considered handled when `debputy` checks for paths you might have forgotten to install.  The `discard`
feature is therefore *also* replaces the `debian/not-installed` file used by `debhelper` and `cdbs`.

# Binary package rules

Inside the manifest, the `packages` mapping can be used to define requests for the binary packages
you want `debputy` to produce.  Each key inside `packages` must be the name of a binary package
defined in `debian/control`.  The value is a dictionary defining which features that `debputy`
should apply to that binary package.  An example could be:


    packages:
        foo:
            transformations:
                - create-symlink:
                      path: usr/share/foo/my-first-symlink
                      target: /usr/share/bar/symlink-target
                - create-symlink:
                      path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
                      target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target
        bar:
            transformations:
            - directories:
               - some/empty/directory.d
               - another/empty/integration-point.d
            - directories:
                 path: a/third-empty/directory.d
                 owner: www-data
                 group: www-data

In this case, `debputy` will create some symlinks inside the `foo` package and some directories for
the `bar` package.  The following subsections define the keys you can use under each binary package.


## Transformations

You can define a `transformations` under the package definition, which is a list a transformation
rules.  An example:

    packages:
        foo:
            transformations:
              - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md'
              - move:
                    source: bar/*
                    target: foo/


Transformations are ordered and are applied in the listed order.  A path can be matched by multiple
transformations; how that plays out depends on which transformations are applied and in which order.
A quick summary:

 - Transformations that modify the file system layout affect how path matches in later transformations.
   As an example, `move` and `remove` transformations affects what globs and path matches expand to in
   later transformation rules.

 - For other transformations generally the latter transformation overrules the earlier one, when they
   overlap or conflict.

### Remove transformation rule (`remove`)

The remove transformation rule is mostly only useful for single binary source packages, where
everything from upstream's build system is installed automatically into the package.  In those case,
you might find yourself with some files that are _not_ relevant for the Debian package (but would be
relevant for other distros or for non-distro local builds).  Common examples include `INSTALL` files
or `LICENSE` files (when they are just a subset of `debian/copyright`).

In the manifest, you can ask `debputy` to remove paths from the debian package by using the `remove`
transformation rule.  An example being:

    packages:
        foo:
            transformations:
              - remove: 'usr/share/doc/{{PACKAGE}}/INSTALL.md'


The value for `remove` is a string, a list of strings or mapping. When it is a mapping, the mapping
has the following key/value pairs:

* `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
  path(s) inside the package that should be removed. The path match(es) can use globs.
    - When a symlink is matched, then the symlink (not its target) is removed as-is.  When a directory is
      matched, then the directory is removed along with all the contents.

* `when` (optional): A condition as defined in [Conditional rules](#Conditional rules).  This condition will
  be combined with the built-in condition provided by these rules (rather than replacing it).

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Note that `remove` removes paths from future glob matches and transformation rules.

The `remove` rule can also be called `rm`.

This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
for p in ${path}; do
  rm -fr "${p}"
done
```

### Move transformation rule (`move`)

The remove transformation rule is mostly only useful for single binary source packages, where
everything from upstream's build system is installed automatically into the package.  In those case,
you might find yourself with some files that need to be renamed to match Debian specific requirements.

This can be done with the `move` transformation rule, which is a rough emulation of the `mv` command
line tool.  An example being:

    packages:
        foo:
            transformations:
              - move:
                    source: bar/*
                    target: foo/
              # Note this example leaves bar/ as an empty directory. Real world usage would probably
              # follow this with an remove rule for bar/ to avoid an empty directory

The value for `move` is a mapping with the following key/value pairs:

 * `source` (required): A path match defining the source path(s) to be renamed.  The value can use globs and
   substitutions.

 * `target` (required): A path defining the target path.  The value *cannot* use globs, but can use substitution.
   If the target ends with a literal `/` (prior to substitution), the target will *always* be a directory.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

There are two basic cases:
 1. If `source` match exactly one path and `target` is not a directory (including it does not exist),
    then `target` is removed (if it existed) and `source` is renamed to `target`.
    - If the `target` is a directory, and you intentionally want to replace it with a non-directory, please
      add an explicit `remove` transformation for the directory prior to this transformation.

 2. If `source` match more than one path *or* `target` is a directory (or is specified with a trailing slash),
    then all matching sources a moved into `target` retaining their basename.  If `target` already contains
    overlapping basenames with `source`, then the transformation rule with abort with an error if the overlap
    contains a directory. Otherwise, any overlapping paths in `target` will be implicitly removed first.

    - If the replacement of a directory is intentional, please add an explicit `remove` rule for it first.
    - In the case that the `source` glob is exotic enough to match two distinct paths with the same basename,
      then `debputy` will reject the transformation on account of it being ambiguous.

In either case, parent directories of `target` is created as necessary as long as it does not trigger a conflict
(or require traversing symlinks or non-directories).  Additionally, the paths matched by `source` will no longer
match anything (since they are now renamed/relocated), which may affect what path future path matches will apply
to.

Note that like `mv`, while the `source` paths are no longer present, their parent directories will remain.

The `move` rule can also be called `mv`.

### Create symlinks transformation rule (`create-symlink`)

Often, the upstream build system will provide the symlinks for you.  However, in some cases, it is useful for
the packager to define distribution specific symlinks. This can be done via the `create-symlink` transformation
rule.  An example of how to do this is:

    packages:
        foo:
            transformations:
            - create-symlink:
                  path: usr/share/foo/my-first-symlink
                  target: /usr/share/bar/symlink-target
            - create-symlink:
                  path: usr/lib/{{DEB_HOST_MULTIARCH}}/my-second-symlink
                  target: /usr/lib/{{DEB_HOST_MULTIARCH}}/baz/symlink-target

The value for the `create-symlink` key is a mapping, which contains the following keys:

 * `path` (required): The path that should be a symlink.  The path may contain substitution variables
   such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs.  Parent directories are implicitly created
   as necessary.
   * Note that if that path has a different definition in the file system, the file system definition is ignored.
     This means you can use this to overwrite a file or a directory with a symlink. 

 * `target` (required): Where the symlink should point to. The target may contain substitution variables
   such as `{{DEB_HOST_MULTIARCH}}` but _cannot_ use globs.  The link target is _not_ required to exist inside
   the package.
   * The `debputy` tool will normalize the target according to the rules of the Debian Policy.  Use absolute
     or relative target at your own preference.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
rm -fr "${path}"
install -o "root" -g "root" -m "755" -d $(dirname "${path}")"
ln -sf "${target}" "${path}"
```

### Create directories transformation rule (`directories`)

NOTE: This transformation is only really needed if you need to create an empty directory somewhere
in your package as an integration point.  All `debputy` transformations will create directories
as required.

In most cases, upstream build systems and `debputy` will create all the relevant directories.  However, in some
rare cases you may want to explicitly define a path to be a directory.  Maybe to silence a linter that is
warning you about a directory being empty, or maybe you need an empty directory that nothing else is creating
for you. This can be done via the `directories` (alias `mkdirs`) transformation rule. An example being:

    packages:
        bar:
            directories:
            - some/empty/directory.d
            - another/empty/integration-point.d
            - path: a/third-empty/directory.d
              owner: www-data
              group: www-data

The value for the `directories` key is either a string, a list of string or a mapping. When it is a
mapping, the mapping has the following key/value pairs:

 * `path` or `paths` (required): A path (`path`) or a list of path (`paths`) defining the
   path(s) inside the package that should be created as directories. The path(es) _cannot_ use globs
   but can use substitution variables.  Parent directories are implicitly created (with owner `root:root`
   and mode `0755` - only explicitly listed directories are affected by the owner/mode options)

 * `owner` (optional, default `root`): Denotes the owner of the directory (but _not_ what is inside the directory).

 * `group` (optional, default `root`): Denotes the group of the directory (but _not_ what is inside the directory).

 * `mode` (optional, default `"0755"`): Denotes the mode of the directory (but _not_ what is inside the directory).
   Note that numeric mode must always be given as a string (i.e., with quotes).  Symbolic mode can be used as well.
   If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the directory's current mode
   (if it already exists) or `0755` if the directory is created by this transformation.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

When the input is a string or a list of string, then that value is used as shorthand for `path`
or `paths` (respectively).

Unless you have a specific need for the mapping form, you are recommended to use the shorthand form of
just listing the directories you want created.


Note that implicitly created directories (by this or other transformations) always have owner `root:root`
and mode `"0755"`.  If you need a directory tree with special ownership/mode, you will have to list all the
directories in that tree explicitly with the relevant attributes OR use `path-metadata` transformation rule
to change their metadata after creation.


This rule behaves roughly like the following shell snippet when applied:

```shell
set -e
for p in ${path}; do
  install -o "root" -g "root" -m "755" -d $(dirname "${path}")"
  install -o "${owner}" -g "${group}" -m "${mode}" -d "${path}"
done
```

### Change path owner/group or mode (`path-metadata`)

The `debputy` normalizes the path metadata (such as ownership and mode) similar to `dh_fixperms`.
For most packages, the default is what you want.  However, in some cases, the package has a special
case or two that `debputy` does not cover.  In that case, you can tell `debputy` to use the metadata
you want by using the `path-metadata` transformation.  An example being:

    packages:
        foo:
            transformations:
              - path-metadata:
                  usr/bin/sudo:
                      mode: "0755"
              - path-metadata:
                  usr/bin/write:
                      group: tty


The value for the `path-metadata` key is a mapping. The mapping has the following key/value pairs:
Each key defines a path or a glob that may


* `path` or `paths` (required): A path match (`path`) or a list of path matches (`paths`) defining the
  path(s) inside the package that should be affected. The path match(es) can use globs and substitution
  variables. Special-rules for matches:
    - Symlinks are never followed and will never be matched by this rule.
    - Directory handling depends on the `recursive` attribute.

 * `owner` (conditional): Denotes the owner of the paths matched by `path` or `paths`. When omitted, no change of
   owner is done.

 * `group` (conditional): Denotes the group of the paths matched by `path` or `paths`. When omitted, no change of
   group is done.

 * `mode` (conditional): Denotes the mode of the paths matched by `path` or `paths`. When omitted, no change in
   mode is done. Note that numeric mode must always be given as a string (i.e., with quotes).  Symbolic mode can
   be used as well. If symbolic mode uses a relative definition (e.g., `o-rx`), then it is relative to the
   directory's current mode.

 * `recursive` (optional, default `false`): When a directory is matched, then the metadata changes are applied
    to the directory itself. When `recursive` is `true`, then the transformation is *also* applied to all paths
    beneath the directory.

 * `when` (optional): A condition as defined in [Conditional rules](#Conditional rules)

At least one of `owner`, `group` or `mode` must be provided.

This rule behaves roughly like the following shell snippet when applied:

```shell
MAYBE_R_FLAG=$(${recursive} && printf "%s" "-R")

for p in ${path}; do
  if [ -n "${owner}" ] || [ -n "${group}" ]; then
    chown $MAYBE_R_FLAG "${owner}:${group}" "${p}"
  fi
  if [ -n "${mode}" ]; then
    chmod $MAYBE_R_FLAG "${mode}" "${p}"
  fi
done
```

(except all symlinks will be ignored)

## Custom binary version

In the *rare* case that you need a binary package to have a custom version, you can use the `binary-version:`
key to describe the desired package version.  An example being:

    packages:
        foo:
            # The foo package needs a different epoch because we took it over from a different
            # source package with higher epoch version
            binary-version: '1:{{DEB_VERSION_UPSTREAM_REVISION}}'

Use this feature sparingly as it is generally not possible to undo as each version must be monotonously
higher than the previous one.  This feature translates into `-v` option for `dpkg-gencontrol`.

The value for the `binary-version` key is a string that defines the binary version.  Generally, you will
want it to contain one of the versioned related substitution variables such as 
`{{DEB_VERSION_UPSTREAM_REVISION}}`.  Otherwise, you will have to remember to bump the version manually
with each upload as versions cannot be reused and the package would not support binNMUs either.


[reference documentation]: https://documentation.divio.com/reference/
