This document explains the steps related to publishing a version-numbered Drake binary release. It is intended only for use by the few Drake Developer experts who regularly handle the release process.
We publish a minor release approximately once per month in the middle of the
calendar month, with version number is v1.N.0
where N
is monotonically
increasing.
Minor releases
Begin this process around 1 week prior to the intended release date.
The release engineering tools (relnotes, download_release_candidate, push_release, etc.) are supported only on Ubuntu (not macOS).
Prior to release
- Choose the next version number.
- Create a local Drake branch named
release_notes-v1.N.0
(so that others can easily find and push to it after the PR is opened). - As the first commit on the branch, mimic the commit
drake@65adb4dd
in order to disable CI. A quick way to do this might be:git fetch upstream 65adb4dd1f89835ad482d6a9a437cb899c05b779 git cherry-pick FETCH_HEAD
- Push that branch and then open a new pull request titled:
[doc] Add release notes v1.N.0
Make sure that “Allow edits by maintainers” on the GitHub PR page is enabled (the checkbox is checked). Set label
release notes: none
. - For release notes, on an ongoing basis, add recent commit messages to the
release notes draft using the
tools/release_engineering/relnotes
tooling. (Instructions can be found atop its source code:relnotes.py
)- On the first run, use
--action=create
to bootstrap the file.- The output is draft release notes in
doc/_release-notes/v1.N.0.md
.
- The output is draft release notes in
- On the subsequent runs, use
--action=update
to refresh the file.- Try to avoid updating the release notes to refer to changes newer than
the likely release, i.e., if you run
--update
on the morning you’re actually doing the release, be sure to pass the--target_commit=
argument to avoid including commits that will not be part of the tag.
- Try to avoid updating the release notes to refer to changes newer than
the likely release, i.e., if you run
- On the first run, use
- For release notes, on an ongoing basis, clean up and relocate the commit notes to properly organized and wordsmithed bullet points. See Polishing the release notes.
- From time to time, merge
upstream/master
into yourorigin/release_notes-v1.N.0
branch (so that it doesn’t go stale). Never rebase or force-push to the branch. We expect that several people will clone and push to it concurrently. - As the release is nearly ready, post a call for action for feature teams to
look at the draft document and provide suggestions (in reviewable) or fixes
(as pushes).
- To help ensure that the “newly deprecated APIs” section is accurate, grep
the code for
YYYY-MM-01
deprecation notations, for theMM
values that would have been associated with our +3 months typical period.
- To help ensure that the “newly deprecated APIs” section is accurate, grep
the code for
Polishing the release notes
Here are some guidelines for bringing commit notes from the relnotes tool into the main body of the document:
- Many commit messages can be cut down to their summary strings and used as-is.
- File geometry/optimization changes under the “Mathematical Program” heading, not the “Multibody” heading.
- Expand all acronyms (eg, MBP -> MultibodyPlant, SG -> SceneGraph).
- Commits can be omitted if they only affect tests or non-installed examples.
- In general you should mention deprecated/removed classes and methods using
their exact name (for easier searching).
- In the deprecation section you can provide the fully-qualified name as the whole line item; the meaning is clear from context.
- This may mean having a long list of items for a single commit. That is fine.
- We have four common grammatical forms for our commit messages:
- Past tense (“Added new method foo”) is acceptable
- Noun phrase (“Ability to foo the bar”) is acceptable
- Imperative (“Add new method foo”, i.e. PEP-8 style) is acceptable
- Present tense (“Adds new method foo”, i.e. Google styleguide style) is discouraged
- Use exactly the same wording for the boilerplate items:
- Every dependency upgrade line should be “Upgrade libfoobar to latest release 1.2.3” or “Upgrade funrepo to latest commit”.
- Dependencies should be referred to by their
workspace
name. - Only one dependency change per line. Even if both meshcat and meshcat-python were upgraded in the same pull request, they each should get their own line in the release notes.
- Keep all bullet points to one line.
- Using hard linebreaks to stay under 80 columns makes the bullet lists hard to maintain over time.
- Say “macOS” not “Mac” or “Apple” or etc.
- Say “SDFormat” not “SDF” nor “sdf”.
Cutting the release
- Find a plausible nightly build to use:
- Make sure https://drake-jenkins.csail.mit.edu/view/Production/ is clean
- Make sure https://drake-jenkins.csail.mit.edu/view/Nightly%20Production/
has nothing still running (modulo the
*-coverage
builds, which we can ignore) - Open the latest builds from the following builds:
- https://drake-jenkins.csail.mit.edu/view/Packaging/job/linux-jammy-unprovisioned-gcc-cmake-nightly-packaging/
- https://drake-jenkins.csail.mit.edu/view/Packaging/job/linux-noble-unprovisioned-gcc-cmake-nightly-packaging/
- https://drake-jenkins.csail.mit.edu/view/Packaging/job/mac-arm-sonoma-unprovisioned-clang-cmake-nightly-packaging/
- Check the logs for those packaging builds and find the URLs they posted
to (open the latest build, go to “View as plain text”, and search for
drake/nightly/drake-0.0.20
), and find the date. It will beYYYYMMDD
with today’s date (they kick off after midnight). All of the builds should have the same date. If not, wait until the following night. - Use the
tools/release_engineering/download_release_candidate
tool with the--find-git-sha
option to download and verify that all the nightlies are built from the same commit. (It’s usage instructions are atop its source code: download_release_candidate.py.)
- Launch the staging builds for that git commit sha:
- Open the following Jenkins jobs (e.g., each in its own new window, so you can copy-and-paste sha1 and version easily):
- In the upper right, click “log in” (unless you’re already logged in). This will use your GitHub credentials.
- Click “Build with Parameters”.
- Change “sha1” (not “ci_sha1”) to the full git sha corresponding to
v1.N.0
and “release_version” to1.N.0
(no “v”).- If you mistakenly provide the “v” in “release_version”, your build will appear to work, but actually fail 5-6 minutes later.
- Click “Build”; each build will take around an hour, give or take.
- Wait for all staging jobs to succeed. It’s OK to work on release notes finishing touches in the meantime, but do not merge the release notes nor tag the release until all five builds have succeeded.
- Update the release notes to have the
YYYY-MM-DD
we choose.- There is a dummy date 2099-12-31 nearby that should likewise be changed.
- Make sure that the nightly build git sha from the prior steps matches the
newest_commit
whose changes are enumerated in the notes. - Update the github links within
doc/_pages/from_binary.md
to reflect the upcoming v1.N.0.
- Re-enable CI by reverting the commit you added way up above in step 3 of Prior to release.
- Wait for the wheel builds to complete, and then download release artifacts:
- Use the
tools/release_engineering/download_release_candidate
tool with the--version
option to download and verify all binaries. (It’s usage instructions are atop its source code: download_release_candidate.py.)
- Use the
- Merge the release notes PR
- Take care when squashing not to accept github’s auto-generated commit message if it is not appropriate.
- After merge, go to https://drake-jenkins.csail.mit.edu/view/Documentation/job/linux-jammy-unprovisioned-gcc-bazel-nightly-documentation/ and push “Build now”.
- If you don’t have “Build now” click “Log in” first in upper right.
- Open https://github.com/RobotLocomotion/drake/releases and choose “Draft
a new release”. Note that this page has neither history nor undo. Be
slow and careful!
- Tag version is: v1.N.0
- Target is: [the git sha from the
--find-git-sha
in step 1.v]- You should select the commit from Target > Recent Commits. The search via commit does not work if you don’t use the correct length.
- Release title is: Drake v1.N.0
- The body of the release should be forked from the prior release (open the
prior release’s web page and click “Edit” to get the markdown), with
appropriate edits as follows:
- The version number
- Click the box labeled “Attach binaries by dropping them here or selecting
them.” and then choose for upload the 30 release files from
/tmp/drake-release/v1.N.0/...
:- 9: 3
.tar.gz
+ 6 checksums - 6: 2
.deb
+ 4 checksums - 9: 3 linux
.whl
+ 6 checksums - 6: 2 macOS arm
.whl
+ 4 checksums - Note that on Jammy with
snap
provided Firefox, drag-and-drop from Nautilus will fail, and drop all of your release page inputs typed so far. Use the Firefox-provided selection dialog instead, by clicking on the box.
- 9: 3
- Choose “Save draft” and take a deep breath.
- Once the documentation build finishes, release!
- Check that the link to drake.mit.edu docs from the GitHub release draft page actually works.
- Click “Publish release”
- Notify
@BetsyMcPhail
by creating a GitHub issue asking her to manually tag docker images and upload the releases to S3. Be sure to provide her with the release tag in the same ping. - Create a GitHub issue on the drake-ros
repository, requesting an update of the
DRAKE_SUGGESTED_VERSION
constant. - Announce on Drake Slack,
#general
. - Party on, Wayne.
Post-release wheel upload
After tagging the release, you must manually upload a PyPI release.
If you haven’t done so already, follow Drake’s PyPI account setup instructions to obtain a username and password.
Most likely, you will want to use an api token to authenticate yourself to the
twine
uploader. See https://pypi.org/help/#apitoken and https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#create-an-account
for advice on managing api tokens.
For Jammy (and later?), apt install twine
gives a perfectly adequate
version of twine
.
-
Run
twine
to upload the wheel release, as follows:- You will need your PyPI username and password for this. (Do not use drake-robot.)
- Run
twine upload /tmp/drake-release/v1.N.0/*.whl
. - Confirm that all of the uploads succeeded without any error messages in your terminal.
Post-release tutorials updates
Upgrade our Deepnote-hosted tutorials to the latest release. This requires
that you have “Edit” permission in the Deepnote project. If you don’t have
that yet, then ask for help on slack in the #releases
channel. Provide
the email address associated with your github account.
- Post a new slack thread in
#releases
saying that you’re beginning the tutorials deployment now (so that others are aware of the potentially- disruptive changes). - Open the tutorials Dockerfile:
- Edit the first line to refer to the YYYYMMDD for this release (login
with your github account; otherwise, the file is read-only).
- For reference, the typical content is thus:
FROM robotlocomotion/drake:jammy-20241114 RUN apt-get -q update && apt-get -q install -y --no-install-recommends nginx-light && apt-get -q clean ENV PATH="/opt/drake/bin:${PATH}" \ PYTHONPATH="/opt/drake/lib/python3.10/site-packages:${PYTHONPATH}"
- If the current content differs by more than just the date from the
above template, ask for help on slack in the
#releases
channel.
- For reference, the typical content is thus:
- After editing the date, click the “Build” button in the upper right,
and wait for the build to succeed.
- If the build fails due to an infrastructure flake, you’ll need to
tweak the Dockerfile before Deepnote will allow you to re-run the
Build. For example, add
&& true
to the end of a RUN line.
- If the build fails due to an infrastructure flake, you’ll need to
tweak the Dockerfile before Deepnote will allow you to re-run the
Build. For example, add
- Edit the first line to refer to the YYYYMMDD for this release (login
with your github account; otherwise, the file is read-only).
- For reference (no action required), the
requirements.txt
file should have the following content:
ipywidgets==7.7.0
- For reference (no action required), the initialization notebook at
init.ipynb
has this cell added the bottom, as a Drake-specific customization:
%%bash /opt/drake/share/drake/setup/deepnote/install_nginx
For Jammy we also needed to add
cd /work
atop the stanza that checks forrequirements.txt
to get it working again. - Copy the updated tutorials from the pinned Dockerfile release
(in
/opt/drake/share/drake/tutorials/...
) into the Deepnote project storage (~/work/...
):- Open zzz_for_maintainers.ipynb.
- Run each cell one by one, checking for errors as you go.
- Note the first cell will take 1-2 minutes to finish because Deepnote needs to boot the machine first.
- Next you’ll copy and run each notebook (in alphabetical order).
Read all of these instructions before performing any of them.
- Caveats for specific notebook names:
- Do not run
licensed_solvers_deepnote
; you don’t have a license. - Do not re-run
zzz_for_maintainers
; you’ve already done that. - The
authoring_multibody_simulation
notebook will appear to hang on one of the middle cells where it uses JointSliders. It is not hung, rather it is waiting for user input. Find the “Meshcat URL” link earlier in the notebook, click through to open Meshcat in a new tab, click “Open Controls”, then click “Stop JointSliders” repeatedly until the option vanishes and the notebook completes. - The
rendering_multibody_plant
sometimes crashes with an interrupted error. In that case, click through to the “Environment” gear in the left-hand panel, then into theinit.ipynb
notebook and re-run the initialization. Then go back torendering_multibody_plant
and try again.
- Do not run
- To deploy run each of the ~2 dozen notebooks (i.e., do this step for
authoring_leaf_system
thenauthoring_multibody_simulation
then … etc.):- In the left-hand panel of your screen, take note that each notebook appears in two places – in “NOTEBOOKS” near the top and in “FILES” near the bottom. The “NOTEBOOKS” is the older copy (from the prior release); the “FILES” is the new copy (from this release). Our goal is to replace the old copy with the new.
- Scroll down to the “FILES” and choose the top-most name. Right click on it and select “Move to notebooks”. Be patient because the web interface could be slow, and there might be delay while copying the file. Note that even though the button says “Move”, it actually only copies the file; it does not delete the item from “FILES”.
- Because a notebook of that name already existed in “NOTEBOOKS” (the old
copy), the moved notebook will be renamed with a
-2
suffix. - Scroll up to “NOTEBOOKS”. Right click on the old copy (without
-2
) and select “Delete” and confirm. Right click on the new notebook (with-2
) and select “Rename” and remove the-2
suffix. - Open the (new) notebook and click “Run notebook”. It should succeed.
- For all code cells, examine the output of each cell to check that no
exceptions have snuck through (or any other unexpected error text).
- The error “‘%matplotlib notebook’ is not supported in Deepnote” is expected and can be ignored.
- For all markdown cells, quickly skim over the rendered output to check that no markup errors have snuck through (e.g., LaTeX syntax errors).
- If you get an error like “Synchronization of file … failed, your changes are not being saved. You might be running out of disk quota” you may ignore it.
- Leave the notebook output intact (do not clear the outputs). We want users to be able to read the outputs on their own, without necessarily running the notebook themselves.
- Go back to “FILES” and right-click then “Delete” on the notebook you
just copied; it should still be the top-most
*.ipynb
in “FILES”.
- Caveats for specific notebook names:
- On the left side, click “Environment” then “Stop Machine”, as a courtesy. (It will time out on its own within the hour, but we might as well save a few nanograms of CO2 where we can.)