From 206ec6e2f694f1e8f0c3a9d37d5273ef0286e821 Mon Sep 17 00:00:00 2001 From: flip1995 Date: Fri, 21 Jan 2022 17:11:00 +0100 Subject: [PATCH] Move syncing doc to book --- CONTRIBUTING.md | 125 -------------------------------- book/src/SUMMARY.md | 1 + book/src/infrastructure/sync.md | 125 ++++++++++++++++++++++++++++++++ 3 files changed, 126 insertions(+), 125 deletions(-) create mode 100644 book/src/infrastructure/sync.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ab2bd59137..85eb82a646c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -21,11 +21,6 @@ All contributors are expected to follow the [Rust Code of Conduct]. - [IntelliJ Rust](#intellij-rust) - [Rust Analyzer](#rust-analyzer) - [How Clippy works](#how-clippy-works) - - [Syncing changes between Clippy and `rust-lang/rust`](#syncing-changes-between-clippy-and-rust-langrust) - - [Patching git-subtree to work with big repos](#patching-git-subtree-to-work-with-big-repos) - - [Performing the sync from `rust-lang/rust` to Clippy](#performing-the-sync-from-rust-langrust-to-clippy) - - [Performing the sync from Clippy to `rust-lang/rust`](#performing-the-sync-from-clippy-to-rust-langrust) - - [Defining remotes](#defining-remotes) - [Issue and PR triage](#issue-and-pr-triage) - [Bors and Homu](#bors-and-homu) - [Contributions](#contributions) @@ -205,126 +200,6 @@ That's why the `else_if_without_else` example uses the `register_early_pass` fun [early_lint_pass]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/trait.EarlyLintPass.html [late_lint_pass]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/trait.LateLintPass.html -## Syncing changes between Clippy and [`rust-lang/rust`] - -Clippy currently gets built with a pinned nightly version. - -In the `rust-lang/rust` repository, where rustc resides, there's a copy of Clippy -that compiler hackers modify from time to time to adapt to changes in the unstable -API of the compiler. - -We need to sync these changes back to this repository periodically, and the changes -made to this repository in the meantime also need to be synced to the `rust-lang/rust` repository. - -To avoid flooding the `rust-lang/rust` PR queue, this two-way sync process is done -in a bi-weekly basis if there's no urgent changes. This is done starting on the day of -the Rust stable release and then every other week. That way we guarantee that we keep -this repo up to date with the latest compiler API, and every feature in Clippy is available -for 2 weeks in nightly, before it can get to beta. For reference, the first sync -following this cadence was performed the 2020-08-27. - -This process is described in detail in the following sections. For general information -about `subtree`s in the Rust repository see [Rust's `CONTRIBUTING.md`][subtree]. - -### Patching git-subtree to work with big repos - -Currently, there's a bug in `git-subtree` that prevents it from working properly -with the [`rust-lang/rust`] repo. There's an open PR to fix that, but it's stale. -Before continuing with the following steps, we need to manually apply that fix to -our local copy of `git-subtree`. - -You can get the patched version of `git-subtree` from [here][gitgitgadget-pr]. -Put this file under `/usr/lib/git-core` (taking a backup of the previous file) -and make sure it has the proper permissions: - -```bash -sudo cp --backup /path/to/patched/git-subtree.sh /usr/lib/git-core/git-subtree -sudo chmod --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree -sudo chown --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree -``` - -_Note:_ The first time running `git subtree push` a cache has to be built. This -involves going through the complete Clippy history once. For this you have to -increase the stack limit though, which you can do with `ulimit -s 60000`. -Make sure to run the `ulimit` command from the same session you call git subtree. - -_Note:_ If you are a Debian user, `dash` is the shell used by default for scripts instead of `sh`. -This shell has a hardcoded recursion limit set to 1000. In order to make this process work, -you need to force the script to run `bash` instead. You can do this by editing the first -line of the `git-subtree` script and changing `sh` to `bash`. - -### Performing the sync from [`rust-lang/rust`] to Clippy - -Here is a TL;DR version of the sync process (all of the following commands have -to be run inside the `rust` directory): - -1. Clone the [`rust-lang/rust`] repository or make sure it is up to date. -2. Checkout the commit from the latest available nightly. You can get it using `rustup check`. -3. Sync the changes to the rust-copy of Clippy to your Clippy fork: - ```bash - # Make sure to change `your-github-name` to your github name in the following command. Also be - # sure to either use a net-new branch, e.g. `sync-from-rust`, or delete the branch beforehand - # because changes cannot be fast forwarded - git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy sync-from-rust - ``` - - _Note:_ This will directly push to the remote repository. You can also push - to your local copy by replacing the remote address with `/path/to/rust-clippy` - directory. - - _Note:_ Most of the time you have to create a merge commit in the - `rust-clippy` repo (this has to be done in the Clippy repo, not in the - rust-copy of Clippy): - ```bash - git fetch origin && git fetch upstream - git checkout sync-from-rust - git merge upstream/master - ``` -4. Open a PR to `rust-lang/rust-clippy` and wait for it to get merged (to - accelerate the process ping the `@rust-lang/clippy` team in your PR and/or - ~~annoy~~ ask them in the [Zulip] stream.) - -### Performing the sync from Clippy to [`rust-lang/rust`] - -All of the following commands have to be run inside the `rust` directory. - -1. Make sure Clippy itself is up-to-date by following the steps outlined in the previous -section if necessary. - -2. Sync the `rust-lang/rust-clippy` master to the rust-copy of Clippy: - ```bash - git checkout -b sync-from-clippy - git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master - ``` -3. Open a PR to [`rust-lang/rust`] - -### Defining remotes - -You may want to define remotes, so you don't have to type out the remote -addresses on every sync. You can do this with the following commands (these -commands still have to be run inside the `rust` directory): - -```bash -# Set clippy-upstream remote for pulls -$ git remote add clippy-upstream https://github.com/rust-lang/rust-clippy -# Make sure to not push to the upstream repo -$ git remote set-url --push clippy-upstream DISABLED -# Set clippy-origin remote to your fork for pushes -$ git remote add clippy-origin git@github.com:your-github-name/rust-clippy -# Set a local remote -$ git remote add clippy-local /path/to/rust-clippy -``` - -You can then sync with the remote names from above, e.g.: - -```bash -$ git subtree push -P src/tools/clippy clippy-local sync-from-rust -``` - -[gitgitgadget-pr]: https://github.com/gitgitgadget/git/pull/493 -[subtree]: https://rustc-dev-guide.rust-lang.org/contributing.html#external-dependencies-subtree -[`rust-lang/rust`]: https://github.com/rust-lang/rust - ## Issue and PR triage Clippy is following the [Rust triage procedure][triage] for issues and pull diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 5d10101d637..200b0cb29ff 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -22,6 +22,7 @@ - [Adding Lints](development/adding_lints.md) - [Common Tools](development/common_tools_writing_lints.md) - [Infrastructure](infrastructure/README.md) + - [Syncing changes between Clippy and rust-lang/rust](infrastructure/sync.md) - [Backporting Changes](infrastructure/backport.md) - [Updating the Changelog](infrastructure/changelog_update.md) - [Release a New Version](infrastructure/release.md) diff --git a/book/src/infrastructure/sync.md b/book/src/infrastructure/sync.md new file mode 100644 index 00000000000..981e63cfdda --- /dev/null +++ b/book/src/infrastructure/sync.md @@ -0,0 +1,125 @@ +# Syncing changes between Clippy and [`rust-lang/rust`] + +Clippy currently gets built with a pinned nightly version. + +In the `rust-lang/rust` repository, where rustc resides, there's a copy of +Clippy that compiler hackers modify from time to time to adapt to changes in the +unstable API of the compiler. + +We need to sync these changes back to this repository periodically, and the +changes made to this repository in the meantime also need to be synced to the +`rust-lang/rust` repository. + +To avoid flooding the `rust-lang/rust` PR queue, this two-way sync process is +done in a bi-weekly basis if there's no urgent changes. This is done starting on +the day of the Rust stable release and then every other week. That way we +guarantee that we keep this repo up to date with the latest compiler API, and +every feature in Clippy is available for 2 weeks in nightly, before it can get +to beta. For reference, the first sync following this cadence was performed the +2020-08-27. + +This process is described in detail in the following sections. For general +information about `subtree`s in the Rust repository see [Rust's +`CONTRIBUTING.md`][subtree]. + +## Patching git-subtree to work with big repos + +Currently, there's a bug in `git-subtree` that prevents it from working properly +with the [`rust-lang/rust`] repo. There's an open PR to fix that, but it's +stale. Before continuing with the following steps, we need to manually apply +that fix to our local copy of `git-subtree`. + +You can get the patched version of `git-subtree` from [here][gitgitgadget-pr]. +Put this file under `/usr/lib/git-core` (making a backup of the previous file) +and make sure it has the proper permissions: + +```bash +sudo cp --backup /path/to/patched/git-subtree.sh /usr/lib/git-core/git-subtree +sudo chmod --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree +sudo chown --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subtree +``` + +> _Note:_ The first time running `git subtree push` a cache has to be built. +> This involves going through the complete Clippy history once. For this you +> have to increase the stack limit though, which you can do with `ulimit -s +> 60000`. Make sure to run the `ulimit` command from the same session you call +> git subtree. + +> _Note:_ If you are a Debian user, `dash` is the shell used by default for +> scripts instead of `sh`. This shell has a hardcoded recursion limit set to +> 1000. In order to make this process work, you need to force the script to run +> `bash` instead. You can do this by editing the first line of the `git-subtree` +> script and changing `sh` to `bash`. + +## Performing the sync from [`rust-lang/rust`] to Clippy + +Here is a TL;DR version of the sync process (all of the following commands have +to be run inside the `rust` directory): + +1. Clone the [`rust-lang/rust`] repository or make sure it is up to date. +2. Checkout the commit from the latest available nightly. You can get it using + `rustup check`. +3. Sync the changes to the rust-copy of Clippy to your Clippy fork: + ```bash + # Make sure to change `your-github-name` to your github name in the following command. Also be + # sure to either use a net-new branch, e.g. `sync-from-rust`, or delete the branch beforehand + # because changes cannot be fast forwarded and you have to run this command again. + git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy sync-from-rust + ``` + + > _Note:_ This will directly push to the remote repository. You can also + > push to your local copy by replacing the remote address with + > `/path/to/rust-clippy` directory. + + > _Note:_ Most of the time you have to create a merge commit in the + > `rust-clippy` repo (this has to be done in the Clippy repo, not in the + > rust-copy of Clippy): + ```bash + git fetch upstream # assuming upstream is the rust-lang/rust remote + git checkout sync-from-rust + git merge upstream/master + ``` +4. Open a PR to `rust-lang/rust-clippy` and wait for it to get merged (to + accelerate the process ping the `@rust-lang/clippy` team in your PR and/or + ask them in the [Zulip] stream.) + +[Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/clippy + +## Performing the sync from Clippy to [`rust-lang/rust`] + +All of the following commands have to be run inside the `rust` directory. + +1. Make sure you have checked out the latest `master` of `rust-lang/rust`. +2. Sync the `rust-lang/rust-clippy` master to the rust-copy of Clippy: + ```bash + git checkout -b sync-from-clippy + git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master + ``` +3. Open a PR to [`rust-lang/rust`] + +## Defining remotes + +You may want to define remotes, so you don't have to type out the remote +addresses on every sync. You can do this with the following commands (these +commands still have to be run inside the `rust` directory): + +```bash +# Set clippy-upstream remote for pulls +$ git remote add clippy-upstream https://github.com/rust-lang/rust-clippy +# Make sure to not push to the upstream repo +$ git remote set-url --push clippy-upstream DISABLED +# Set clippy-origin remote to your fork for pushes +$ git remote add clippy-origin git@github.com:your-github-name/rust-clippy +# Set a local remote +$ git remote add clippy-local /path/to/rust-clippy +``` + +You can then sync with the remote names from above, e.g.: + +```bash +$ git subtree push -P src/tools/clippy clippy-local sync-from-rust +``` + +[gitgitgadget-pr]: https://github.com/gitgitgadget/git/pull/493 +[subtree]: https://rustc-dev-guide.rust-lang.org/contributing.html#external-dependencies-subtree +[`rust-lang/rust`]: https://github.com/rust-lang/rust -- 2.44.0