*: Automate various parts of the release process

[thomaseizinger]

Description

Most tasks around releasing crates in our monorepo are currently performed manually:

• Adding changelog entries (happens in PRs usually)
• Pull through version numbers of workspace dependencies (also happens in PRs)
• Actually releasing crates to crates.io (done by @mxinden)

Motivation

Repetitive tasks consume unnecessary time and should be automated.

Requirements

1. I'd like to have the pulling through of version bumps somehow automated, i.e. if I bump libp2p-core by a minor version, all dependent crates should update their dependency automatically and bump their on version as required.
2. Adding a changelog entry about the above would be useful.
3. Automatically lint against broken links in markdown. This is currently not possible because the root CHANGELOG links to what the header will be after the release, meaning until then, they are all broken because the contain the -unreleased postfix.

It would be nice to trigger these operations through a bot / CI so contributors don't have to learn how to do it.

Open questions

Are you planning to do it yourself in a pull request?

Maybe.

[thomaseizinger]

I would suggest we make use of the cargo-xtask pattern: https://github.com/matklad/cargo-xtask/

[BigLep]

It's great to see this issue.

I think the first step here is to document today what we do. I don't best (automation) to be the enemy of better (document what we do). Basically I view the first step is "repeatability". Then we can move to automating more and more of it. We need to make sure this isn't trible knowledge that only lives in a limited number of peoples' heads.

@mxinden : in libp2p/test-plans#41 (comment), you raised the question on the priority of this. I'd like to prevent this from being an "either or" conversation. I think it should be a "both and". Something is better than nothing on this front.

A couple of options for moving forward:

1. the next time you do a release, have someone else shepherd you the whole time. Maybe even do a screen recording. It's their responsibility to then write up the steps if you can't do it. The next time you reverse shadow to see where the gaps are.
2. (better I think) You take first stab at writing the steps. For the next release, you have someone else attempt to follow them. You are shadowing the process in case they get stuck. The release is then responsible for fixing any issues in the steps that see.

I think option 2 is better because it gets us validating the correctness/completeness of the release process quicker, but I defer to the team.

Other ideas:

1. @p-shahi may be able to help, but I think it's also great if rust-libp2p maintainers move the ball forward independently as well.
2. Maybe this also plays well in the Little Bear Labs camp of them having Rust expertise and wanting to work towards sharing more of the maintenance of the project.

[thomaseizinger]

I've played around with Google's release-please and it works quite well. Here is an example of a release PR: thomaseizinger#5

• It builds on the conventional commit spec. This should be easy for us because we squash commits, so as the maintainers, we can ensure those conventions are properly enforces as we merge a PR.
• It supports¹ cargo workspaces and will automatically bump dependent crates accordingly. I think that it only patch-bumps those so we'd need to check if we can somehow enforce a minor/major bump for the crates where we know that a dependency bump is a breaking change. This is the case for dependencies such as libp2p-ping -> libp2p-swarm for example.

Footnotes

1. There is an issue at the moment with our specific setup but I think that should be able to get resolved. ↩

[mxinden]

I took a look at cargo-release. As far as I can tell from a small dry-run cargo-release could simplify the last step of a release, namely the publishing of the many crates in dependency-then-dependent order. It would automate the cargo pubish and git tag steps.

[thomaseizinger]

I took a look at cargo-release. As far as I can tell from a small dry-run cargo-release could simplify the last step of a release, namely the publishing of the many crates in dependency-then-dependent order. It would automate the cargo pubish and git tag steps.

Well that is already something!
If we manage to get release-please working then that would cover everything I think?

[mxinden]

For the record, I did the v0.49.0 release with cargo release. #3028 updates our release documentation accordingly.

This tool makes my life A LOT easier. Wonderful.

[thomaseizinger]

For the record, I did the v0.49.0 release with cargo release. #3028 updates our release documentation accordingly.

This tool makes my life A LOT easier. Wonderful.

That is great! What do you think of making a workflow that does what you did every time a release/x.y.z branch is merged into master?

This should be helpful, regardless of how we create that release branch, i.e. manually or with "release-please" or another tool.

[mxinden]

I am in favor of fully automating this. Do we know of any projects that do this today? (I am asking as I don't want rust-libp2p to be special here.)

I don't have the capacity to work on this any time soon. In the meantime, I am fine continuing to do the releases.

[BigLep]

I haven't been following along in all the conversation, but I think it's good hygiene for a project with multiple maintainers to have a document that is clear about "how we do a release". If that document consolidates down to "run this script and you're done", that's great. I don't want us to put off expressing what we do today while we wait for the automated solution.

The reasons for this ask:

1. It fits within our tenets of taking security, stability, performance, and specs seriously. It doesn't directly map to one of those, but it fits in the "aura". We want consumers walking away with a sense of "wow, this project takes the quality of their work very seriously."
2. I want to make sure we have the place to hook additional steps. For example, if we need to have a step for "run this big interoperability test suite to confirm no regressions before releasing because it doesn't run automatically in CI" we want to have a place to inject and show that we did that step. Similarly, if there are marketing-related pieces that we do for a release (e.g., forum post, twitter post, blog entry) that aren't going to be automated in the short term, we want a place to add those.
3. it also gives visibility into how much work is involved with doing a release and how much potential return on investment for simplifying/automating parts. This document doesn't have to be much, but it should be accurate to what we're doing today.

[thomaseizinger]

I think the document we have is up to date with our release process.

I do think we should change it though. All the manual steps like changelog entries and version bumps need to go away. It slows down my own work, esp. right after a release when all the versions are still untouched. It is also an inconvenience for contributors as we need to babysit them to make the correct changelog entries and version bumps.

I am gonna work towards adopting release-please. For that, we need to fix our circular dependencies which conveniently should also improve incremental compile-times.

Anyone against adopting conventional commits? That would be required to automate the changelogs and detect the correct version bumps.

[BigLep]

Thanks @thomaseizinger . I'm certainly supportive of automating and making things simpler.

My bad (and it may just be me since I'm not making rust-libp2p commits) but I only became aware that we have a written document when you alluded to it. If anyone else is following along, the release process document is https://github.com/libp2p/rust-libp2p/blob/master/docs/release.md

Concerning conventional commits, you may want to set a deadline by when folks need to respond by if they are against the idea, or otherwise move forward (i.e., no response means support).

I assume this is known, but I didn't see it mentioned here: js-libp2p is using release-please I believe.

@mxinden : I may be out of the loop, but doesn't our release process need a checklist item for confirming libp2p/test-plan's "big interoperability test" passes before release?

[thomaseizinger]

I assume this is known, but I didn't see it mentioned here: js-libp2p is using release-please I believe.

I did not know this! Thanks for mentioning it. @achingbrain @MarcoPolo Can you comment on this?

Concerning conventional commits, you may want to set a deadline by when folks need to respond by if they are against the idea, or otherwise move forward (i.e., no response means support).

Thanks for the encouragement. Whilst I think it is a good idea, I believe we should properly challenge it first. I'll open a separate issue to discuss: #3054

@mxinden : I may be out of the loop, but doesn't our release process need a checklist item for confirming libp2p/test-plan's "big interoperability test" passes before release?

We run all interoperability tests on each pull request and (typically) only merge when they are green: https://github.com/libp2p/rust-libp2p/actions/workflows/interop-test.yml

[BigLep]

Thanks @thomaseizinger . Concenring interoperability tests, my understanding from a month ago was that what we run in CI is a subset of the full interoperability testsuite (because the full suite takes too long).

@laurentsenta : I'm sorry to pull you in here. I believe this has been discussed before. I looked in https://github.com/libp2p/test-plans/issues for a discussion on this, but didn't find one. Please correct me too if there is no longer a "big tests" that should be run manually for a release.

[mxinden]

Correct. We currently only run the go-rust-interop-latest composition test instead of the go-rust-interop composition test.

rust-libp2p/.github/workflows/interop-test.yml

Lines 26 to 32 in b42f286

	run-ping-interop-cross-implementation:
	uses: "libp2p/test-plans/.github/workflows/run-composition.yml@master"
	with:
	composition_file: "ping/_compositions/go-rust-interop-latest.toml"
	custom_git_target: github.com/${{ github.event.pull_request.head.repo.full_name || github.event.repository.full_name }}
	custom_git_reference: ${{ github.event.pull_request.head.sha || github.sha }}
	custom_interop_target: rust

https://github.com/libp2p/test-plans/blob/master/ping/_compositions/go-rust-interop.toml

@laurentsenta do you know off the top of your head how much longer the full test suite would take? If it is not too much time, my preference would be to simply run the full one on rust-libp2p's pull request CI instead of adding a new step to the release process.