From 3279dec0b542fec2f21c39cb4f5560a80979d4f5 Mon Sep 17 00:00:00 2001 From: Karl Smeltzer Date: Tue, 21 Oct 2025 12:03:51 -0700 Subject: [PATCH 1/2] move generic tooling out of rust faq --- .../setup/set-up-for-ubuntu-development.md | 88 ++++++++++++++++++- .../rustc/rust-packaging-faq.md | 88 +------------------ 2 files changed, 89 insertions(+), 87 deletions(-) diff --git a/docs/contributors/setup/set-up-for-ubuntu-development.md b/docs/contributors/setup/set-up-for-ubuntu-development.md index d35f87b4..9adb1118 100644 --- a/docs/contributors/setup/set-up-for-ubuntu-development.md +++ b/docs/contributors/setup/set-up-for-ubuntu-development.md @@ -74,7 +74,7 @@ other entities. gpg> adduid Real name: Karl Stenerud Email address: karl.stenerud@canonical.com - Comment: + Comment: You selected this USER-ID: "Karl Stenerud " @@ -369,6 +369,92 @@ For more info, see the [LXD documentation](https://documentation.ubuntu.com/lxd/latest/) ``` +(understanding-tooling)= +## Understanding the tooling + +The ecosystem of Debian packaging tools is big and diverse, with a great deal of overlap and intended deprecation. +This section introduces some of the most common tools used to generate Ubuntu packages. +A much more comprehensive reference is available in [the Debian developers-reference](https://www.debian.org/doc/manuals/developers-reference/tools.en.html). + +There are, roughly speaking, three phases to creating a `.deb` package. + +1. Create the source package. +2. Build the binary package from the source package. +3. Test and upload the binary package. + +Although coarse, keeping this in mind can help build a more coherent mental model of how the tools interoperate. + +### Tools for working on the source package + +`uscan` +: Short for "upstream scan", it automates tasks related to keeping up-to-date with upstream code. +It relies on you creating a `debian/watch` file, and then can look for new releases, grab them, and verify signatures. +Source packages require "orig" tarballs as a component, which are the unmodified upstream source code, and `uscan` can help you grab them. + +`quilt` +: An older, but still widely-used tool to manage a sequence of patches that apply to the upstream source. +It can not only help you manage the order of patches and their current state of application, but can help you generate patches from modified source code. +Some maintainers prefer to manage patches with git-based tooling, which didn't exist when `quilt` was first developed. +In theory, a `git` branch could encode the order of the patches, which are then rebased onto the new code when doing an update. +Nevertheless, being familiar with `quilt` is helpful when understanding packages worked on by multiple maintainers. + +`dch` +: Automation for managing the `/debian/changelog` file. +It helps you generate timestamps, author information, and more. + +`git-ubuntu` +: An extension to `git` that can clone the repository attached to a package directly from Launchpad. +It has many other features intended to enable more complex package workflows, which are described in the [Ubuntu Maintainer's Handbook](https://github.com/canonical/ubuntu-maintainers-handbook). + +`pull-lp-source` +: Similar to `git-ubuntu`, which should be used instead where possible. +The main difference is that instead of cloning the `git` repository, it grabs the source package from Launchpad. +That means you'll get things like the orig tarball, but not the `git` history. + +### Tools for building the binary package from the source package + +`sbuild` +: The primary tool Ubuntu Rust packagers will likely use to build binary packages. +Fundamentally it is an orchestration tool and replaces the direct invocation of many other tools. +It creates clean build environments using `schroot`s as a sandbox, containing all the necessary dependencies already installed. +It then invokes `dpkg-buildpackage`, and copies the output out of the `schroot`. +There is an alternative backend for `sbuild`, which replaces `schroot`, called `unshare`. +At the time of writing, Launchpad still uses `schroot` and it is probably preferable to do the same locally to avoid behavior differences. + +`dpkg-buildpackage` +: The lower-level build tool that is invoked for you by `sbuild`. +It is responsible for running all of the scripts defined in `debian/rules`. + +`debhelper` +: A suite of tools used to simplify many of the elements that packages often have in common. +It defines a clear sequence of steps that packages go through when building, which can all be overridden as necessary within `/debian/rules`. +Giving names and order to these steps means that packages no longer need custom scripts to do things like apply patches, or perform configuration. +If you see a call to `dh` in `/debian/rules`, that package is using `debhelper`. +Some older packages may not have migrated to `debhelper` from the manual approach yet. +Lots of add-on packages exist, most notably `dh-cargo` which is listed separately. + +`lintian` +: The static analysis tool use to check packages for policy violations, usually executed on your behalf by `sbuild`. +You should run it on your source package, but also on your compiled binary packages, as the results of just one or the other are not comprehensive. + + +### Tools for working with the compiled binary package + +`lintian` (again) +: As mentioned, you should do static analysis of both your source package and your binary package, as the results will differ and both can be useful. + +`autopkgtest` +: The testing framework for `.deb` packages. +It sets up an isolated environment, which can be any of a number of containers or VMs, and runs the tests defined in the `debian/tests` directory. +In the official archives, these tests run automatically as part of CI. +However, for PPAs in a typical Ubuntu workflow, the tests are not triggered automatically. + +`dput` +: Upload your package to the package archive or a PPA. + +`ppa-dev-tools` +: This collection of tools makes it easy to trigger autopkgtests on Launchpad for a PPA, and to otherwise administer your PPAs. + (caching-packages)= ## Caching packages diff --git a/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md b/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md index 3c64be80..b8990b5c 100644 --- a/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md +++ b/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md @@ -148,48 +148,8 @@ Ubuntu packages Firefox as a Snap, which avoids the need for special hacks like (debian-rust-tools)= ## What Rust-specific tooling do maintainers use, and how does it fit into the wider ecosystem of packaging tools? -The ecosystem of Debian packaging tools is big and diverse, with a great deal of overlap and intended deprecation. -This section focuses only on those tools that are likely to be useful when working on Rust code. -A much more comprehensive reference is available in [the Debian developers-reference](https://www.debian.org/doc/manuals/developers-reference/tools.en.html). - -There are, roughly speaking, three phases to creating a `.deb` package. - -1. Create the source package. -2. Build the binary package from the source package. -3. Test and upload the binary package. - -Although coarse, keeping this in mind can help build a more coherent mental model of how the tools interoperate. - -### Tools for creating and modifying the source package - -**Language-agnostic tools:** - -`uscan` -: Short for "upstream scan", it automates tasks related to keeping up-to-date with upstream code. -It relies on you creating a `debian/watch` file, and then can look for new releases, grab them, and verify signatures. -Source packages require "orig" tarballs as a component, which are the unmodified upstream source code, and `uscan` can help you grab them. - -`quilt` -: An older, but still widely-used tool to manage a sequence of patches that apply to the upstream source. -It can not only help you manage the order of patches and their current state of application, but can help you generate patches from modified source code. -Some maintainers prefer to manage patches with git-based tooling, which didn't exist when `quilt` was first developed. -In theory, a `git` branch could encode the order of the patches, which are then rebased onto the new code when doing an update. -Nevertheless, being familiar with `quilt` is helpful when understanding packages worked on by multiple maintainers. - -`dch` -: Automation for managing the `/debian/changelog` file. -It helps you generate timestamps, author information, and more. - -`git-ubuntu` -: An extension to `git` that can clone the repository attached to a package directly from Launchpad. -It has many other features intended to enable more complex package workflows, which are described in the [Ubuntu Maintainer's Handbook](https://github.com/canonical/ubuntu-maintainers-handbook). - -`pull-lp-source` -: Similar to `git-ubuntu`, which should be used instead where possible. -The main difference is that instead of cloning the `git` repository, it grabs the source package from Launchpad. -That means you'll get things like the orig tarball, but not the `git` history. - -**Rust-specific tools:** +There are a handful of tools useful for Rust package maintainers, but you should first have a good understanding of the {ref}`general Debian tooling `. +Specifically for Rust packages, you should also be familiar with the following. `debcargo` : A tool written in Rust that attempts to automatically convert a Rust crate into a Debian source package by mapping dependencies to Debian packages and generating important files like `debian/control`, `debian/rules`, and `debian/changelog`. @@ -197,55 +157,11 @@ That means you'll get things like the orig tarball, but not the `git` history. `cargo-debstatus` : Pull the dependencies from a Cargo project, and determine if appropriate versions of them are already available in the package archive. -### Tools for building the binary package from the source package - -**Language-agnostic tools:** - -`sbuild` -: The primary tool Ubuntu Rust packagers will likely use to build binary packages. -Fundamentally it is an orchestration tool and replaces the direct invocation of many other tools. -It creates clean build environments using `schroot`s as a sandbox, containing all the necessary dependencies already installed. -It then invokes `dpkg-buildpackage`, and copies the output out of the `schroot`. -There is an alternative backend for `sbuild`, which replaces `schroot`, called `unshare`. -At the time of writing, Launchpad still uses `schroot` and it is probably preferable to do the same locally to avoid behavior differences. - -`dpkg-buildpackage` -: The lower-level build tool that is invoked for you by `sbuild`. -It is responsible for running all of the scripts defined in `debian/rules`. - -`debhelper` -: A suite of tools used to simplify many of the elements that packages often have in common. -It defines a clear sequence of steps that packages go through when building, which can all be overridden as necessary within `/debian/rules`. -Giving names and order to these steps means that packages no longer need custom scripts to do things like apply patches, or perform configuration. -If you see a call to `dh` in `/debian/rules`, that package is using `debhelper`. -Some older packages may not have migrated to `debhelper` from the manual approach yet. -Lots of add-on packages exist, most notably `dh-cargo` which is listed separately. - -`lintian` -: The static analysis tool use to check packages for policy violations, usually executed on your behalf by `sbuild`. -While it's listed under source tooling, it can also check binary packages. - -**Rust-specific** - `dh-cargo` : A `debhelper` add-on that lets you use `--with-cargo` when calling `dh`. In essence, it enables the direct use of Cargo as a build tool for compiling and linking the code in your package. Note that `dh-cargo` should _not_ be used for applications built only partially with Rust. - -### Tools for working with the binary package - -**Language-agnostic tools:** - -`autopkgtest` -: The testing framework for `.deb` packages. -It sets up an isolated environment, which can be any of a number of containers or VMs, and runs the tests defined in the `debian/tests` directory. -In the official archives, these tests run automatically as part of CI. -However, for PPAs in a typical Ubuntu workflow, the tests are not triggered automatically. - -`dput` -: Upload your package to the package archive or a PPA. - ## What is the relationship between `llvm` and `rustc` packages? The relationship can be one of two things, depending on the particular `rustc` package version. From b4df4b6d01b7edc10ba99797cd22ec4e05e8361b Mon Sep 17 00:00:00 2001 From: Karl Smeltzer Date: Fri, 24 Oct 2025 10:33:26 -0700 Subject: [PATCH 2/2] Move tooling discussion into new page --- docs/contributors/setup/index.md | 1 + .../setup/set-up-for-ubuntu-development.md | 86 ------------------- .../setup/understand-the-tools.md | 85 ++++++++++++++++++ .../rustc/rust-packaging-faq.md | 2 +- 4 files changed, 87 insertions(+), 87 deletions(-) create mode 100644 docs/contributors/setup/understand-the-tools.md diff --git a/docs/contributors/setup/index.md b/docs/contributors/setup/index.md index 06ac6f5a..297ecfb4 100644 --- a/docs/contributors/setup/index.md +++ b/docs/contributors/setup/index.md @@ -7,3 +7,4 @@ Set up for Ubuntu development Get the source of a package Consider PGP key storage recommendations +Understand the tools diff --git a/docs/contributors/setup/set-up-for-ubuntu-development.md b/docs/contributors/setup/set-up-for-ubuntu-development.md index 9adb1118..dc331308 100644 --- a/docs/contributors/setup/set-up-for-ubuntu-development.md +++ b/docs/contributors/setup/set-up-for-ubuntu-development.md @@ -369,92 +369,6 @@ For more info, see the [LXD documentation](https://documentation.ubuntu.com/lxd/latest/) ``` -(understanding-tooling)= -## Understanding the tooling - -The ecosystem of Debian packaging tools is big and diverse, with a great deal of overlap and intended deprecation. -This section introduces some of the most common tools used to generate Ubuntu packages. -A much more comprehensive reference is available in [the Debian developers-reference](https://www.debian.org/doc/manuals/developers-reference/tools.en.html). - -There are, roughly speaking, three phases to creating a `.deb` package. - -1. Create the source package. -2. Build the binary package from the source package. -3. Test and upload the binary package. - -Although coarse, keeping this in mind can help build a more coherent mental model of how the tools interoperate. - -### Tools for working on the source package - -`uscan` -: Short for "upstream scan", it automates tasks related to keeping up-to-date with upstream code. -It relies on you creating a `debian/watch` file, and then can look for new releases, grab them, and verify signatures. -Source packages require "orig" tarballs as a component, which are the unmodified upstream source code, and `uscan` can help you grab them. - -`quilt` -: An older, but still widely-used tool to manage a sequence of patches that apply to the upstream source. -It can not only help you manage the order of patches and their current state of application, but can help you generate patches from modified source code. -Some maintainers prefer to manage patches with git-based tooling, which didn't exist when `quilt` was first developed. -In theory, a `git` branch could encode the order of the patches, which are then rebased onto the new code when doing an update. -Nevertheless, being familiar with `quilt` is helpful when understanding packages worked on by multiple maintainers. - -`dch` -: Automation for managing the `/debian/changelog` file. -It helps you generate timestamps, author information, and more. - -`git-ubuntu` -: An extension to `git` that can clone the repository attached to a package directly from Launchpad. -It has many other features intended to enable more complex package workflows, which are described in the [Ubuntu Maintainer's Handbook](https://github.com/canonical/ubuntu-maintainers-handbook). - -`pull-lp-source` -: Similar to `git-ubuntu`, which should be used instead where possible. -The main difference is that instead of cloning the `git` repository, it grabs the source package from Launchpad. -That means you'll get things like the orig tarball, but not the `git` history. - -### Tools for building the binary package from the source package - -`sbuild` -: The primary tool Ubuntu Rust packagers will likely use to build binary packages. -Fundamentally it is an orchestration tool and replaces the direct invocation of many other tools. -It creates clean build environments using `schroot`s as a sandbox, containing all the necessary dependencies already installed. -It then invokes `dpkg-buildpackage`, and copies the output out of the `schroot`. -There is an alternative backend for `sbuild`, which replaces `schroot`, called `unshare`. -At the time of writing, Launchpad still uses `schroot` and it is probably preferable to do the same locally to avoid behavior differences. - -`dpkg-buildpackage` -: The lower-level build tool that is invoked for you by `sbuild`. -It is responsible for running all of the scripts defined in `debian/rules`. - -`debhelper` -: A suite of tools used to simplify many of the elements that packages often have in common. -It defines a clear sequence of steps that packages go through when building, which can all be overridden as necessary within `/debian/rules`. -Giving names and order to these steps means that packages no longer need custom scripts to do things like apply patches, or perform configuration. -If you see a call to `dh` in `/debian/rules`, that package is using `debhelper`. -Some older packages may not have migrated to `debhelper` from the manual approach yet. -Lots of add-on packages exist, most notably `dh-cargo` which is listed separately. - -`lintian` -: The static analysis tool use to check packages for policy violations, usually executed on your behalf by `sbuild`. -You should run it on your source package, but also on your compiled binary packages, as the results of just one or the other are not comprehensive. - - -### Tools for working with the compiled binary package - -`lintian` (again) -: As mentioned, you should do static analysis of both your source package and your binary package, as the results will differ and both can be useful. - -`autopkgtest` -: The testing framework for `.deb` packages. -It sets up an isolated environment, which can be any of a number of containers or VMs, and runs the tests defined in the `debian/tests` directory. -In the official archives, these tests run automatically as part of CI. -However, for PPAs in a typical Ubuntu workflow, the tests are not triggered automatically. - -`dput` -: Upload your package to the package archive or a PPA. - -`ppa-dev-tools` -: This collection of tools makes it easy to trigger autopkgtests on Launchpad for a PPA, and to otherwise administer your PPAs. - (caching-packages)= ## Caching packages diff --git a/docs/contributors/setup/understand-the-tools.md b/docs/contributors/setup/understand-the-tools.md new file mode 100644 index 00000000..d5d0900a --- /dev/null +++ b/docs/contributors/setup/understand-the-tools.md @@ -0,0 +1,85 @@ +(understand-the-tools)= +# Understand the tools + +The ecosystem of Debian packaging tools is big and diverse, with a great deal of overlap and intended deprecation. +This section introduces some of the most common tools used to generate Ubuntu packages. +A much more comprehensive reference is available in [the Debian developers-reference](https://www.debian.org/doc/manuals/developers-reference/tools.en.html). + +There are, roughly speaking, three phases to creating a `.deb` package. + +1. Create the source package. +2. Build the binary package from the source package. +3. Test and upload the binary package. + +Although coarse, keeping this in mind can help build a more coherent mental model of how the tools interoperate. + +## Tools for working on the source package + +`uscan` +: Short for "upstream scan", it automates tasks related to keeping up-to-date with upstream code. +It relies on you creating a `debian/watch` file, and then can look for new releases, grab them, and verify signatures. +Source packages require "orig" tarballs as a component, which are the unmodified upstream source code, and `uscan` can help you grab them. + +`quilt` +: An older, but still widely-used tool to manage a sequence of patches that apply to the upstream source. +It can not only help you manage the order of patches and their current state of application, but can help you generate patches from modified source code. +Some maintainers prefer to manage patches with git-based tooling, which didn't exist when `quilt` was first developed. +In theory, a `git` branch could encode the order of the patches, which are then rebased onto the new code when doing an update. +Nevertheless, being familiar with `quilt` is helpful when understanding packages worked on by multiple maintainers. + +`dch` +: Automation for managing the `/debian/changelog` file. +It helps you generate timestamps, author information, and more. + +`git-ubuntu` +: An extension to `git` that can clone the repository attached to a package directly from Launchpad. +It has many other features intended to enable more complex package workflows, which are described in the [Ubuntu Maintainer's Handbook](https://github.com/canonical/ubuntu-maintainers-handbook). + +`pull-lp-source` +: Similar to `git-ubuntu`, which should be used instead where possible. +The main difference is that instead of cloning the `git` repository, it grabs the source package from Launchpad. +That means you'll get things like the orig tarball, but not the `git` history. + +## Tools for building the binary package from the source package + +`sbuild` +: The primary tool Ubuntu Rust packagers will likely use to build binary packages. +Fundamentally it is an orchestration tool and replaces the direct invocation of many other tools. +It creates clean build environments using `schroot`s as a sandbox, containing all the necessary dependencies already installed. +It then invokes `dpkg-buildpackage`, and copies the output out of the `schroot`. +There is an alternative backend for `sbuild`, which replaces `schroot`, called `unshare`. +At the time of writing, Launchpad still uses `schroot` and it is probably preferable to do the same locally to avoid behavior differences. + +`dpkg-buildpackage` +: The lower-level build tool that is invoked for you by `sbuild`. +It is responsible for running all of the scripts defined in `debian/rules`. + +`debhelper` +: A suite of tools used to simplify many of the elements that packages often have in common. +It defines a clear sequence of steps that packages go through when building, which can all be overridden as necessary within `/debian/rules`. +Giving names and order to these steps means that packages no longer need custom scripts to do things like apply patches, or perform configuration. +If you see a call to `dh` in `/debian/rules`, that package is using `debhelper`. +Some older packages may not have migrated to `debhelper` from the manual approach yet. +Lots of add-on packages exist, most notably `dh-cargo` which is listed separately. + +`lintian` +: The static analysis tool use to check packages for policy violations, usually executed on your behalf by `sbuild`. +You should run it on your source package, but also on your compiled binary packages, as the results of just one or the other are not comprehensive. + + +## Tools for working with the compiled binary package + +`lintian` (again) +: As mentioned, you should do static analysis of both your source package and your binary package, as the results will differ and both can be useful. + +`autopkgtest` +: The testing framework for `.deb` packages. +It sets up an isolated environment, which can be any of a number of containers or VMs, and runs the tests defined in the `debian/tests` directory. +In the official archives, these tests run automatically as part of CI. +However, for PPAs in a typical Ubuntu workflow, the tests are not triggered automatically. + +`dput` +: Upload your package to the package archive or a PPA. + +`ppa-dev-tools` +: This collection of tools makes it easy to trigger autopkgtests on Launchpad for a PPA, and to otherwise administer your PPAs. diff --git a/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md b/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md index b8990b5c..89efb6c1 100644 --- a/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md +++ b/docs/maintainers/niche-package-maintenance/rustc/rust-packaging-faq.md @@ -148,7 +148,7 @@ Ubuntu packages Firefox as a Snap, which avoids the need for special hacks like (debian-rust-tools)= ## What Rust-specific tooling do maintainers use, and how does it fit into the wider ecosystem of packaging tools? -There are a handful of tools useful for Rust package maintainers, but you should first have a good understanding of the {ref}`general Debian tooling `. +There are a handful of tools useful for Rust package maintainers, but you should first have a good understanding of the {ref}`general Debian tooling `. Specifically for Rust packages, you should also be familiar with the following. `debcargo`