linux kernel

Use b4 for kernel contributions

Posted by Marcus Folkesson on Wednesday, August 23, 2023

Use b4 for kernel contributions

There is a little tool called b4 [1] that has been part of my workflow with the Linux kernel for a while. It's developed to be a tool used to simplify the work of the maintainers, but my main use of the tool has been to fetch patch series from the mailing list and apply them to my local git repository during reviews. I recently noticed that it got a lot of handy features (experimental though) for the contributors as well, which I now want to test!

The reason I started digging into my toolbox started with a discussion I had with a friend about how I could possibly prefer an email-based workflow when working with FOSS versus new and fresh web-based tools like Github and Gitlab. In web-based tools, all you have to do is push the right button, and email is so old-fashioned, isn't it?

First of all, I never hit the right button. Never. Secondly, it disturbs my workflow.

That is also my biggest reason; these "old school" tools fits my workflow perfectly:

  • I use mutt [4] to read and send my emails.
  • I use vim [5] for all text manipulation.
  • I use git [6] to manage my code.
  • I use b4 [1] in the (kernel) review process (and in contributions from now on)

These are the tools that I use for almost every email based FOSS project that I'm involved with. The first three is rather common, but what about b4? It's not too much information out there so I think a little introduction might be good.

/media/b4.png

So, what is b4?

The project started as a tool named get-lore-mbox [2] , which later became b4. Fun fact is that the name, b4, was chosen for ease of typing and because B-4 was the precursor to Lore and Data in the Star Trek universe :-)

B4 is a tool to simplify the development workflow with distributed patches, especially mailing lists. It works with public inbox archives and aims to be a useful tool for both maintainers and developers.

Example on what b4 will do for you:

  • Retrieve patch series from a public mailing list (e.g. lore.kernel.org)
  • Compare patch series
  • Apply patch series to your git repository
  • Prepare and send in your work
  • Retrieve code-review trailers

It's a pretty competent tool.

Install b4

First we have to install b4 on our system.

B4 is propbably already available in your distribution:

Archlinux:

$ pacman -Sy b4

Ubuntu:

$ apt-get install b4

Fedora:

$ dnf install b4

Or whatever package manager you use.

It's also possible to install it with pip:

$ python3 -m pip install --user b4

And of course, run it directly from the git repository [3] :

$ git clone https://git.kernel.org/pub/scm/utils/b4/b4.git
$ cd b4
$ git submodule update --init
$ pip install --user -r requirements.txt

Review patches workflow

I use b4 shazam to fetch the latest version of a patch series and apply them to my tree. All you need to provide is the Message-ID of the thread that you find in the email header of the patch. For instance:

 1$ b4 shazam 20230820102610.755188-6-marcus.folkesson@gmail.com
 2Grabbing thread from lore.kernel.org/all/20230820102610.755188-6-marcus.folkesson@gmail.com/t.mbox.gz
 3Checking for newer revisions
 4Grabbing search results from lore.kernel.org
 5  Added from v8: 7 patches
 6Analyzing 17 messages in the thread
 7Will use the latest revision: v8
 8You can pick other revisions using the -vN flag
 9Checking attestation on all messages, may take a moment...
10---
11  [PATCH v8 1/6] dt-bindings: iio: adc: mcp3911: add support for the whole MCP39xx family
12  [PATCH v8 2/6] iio: adc: mcp3911: make use of dev_err_probe()
13  [PATCH v8 3/6] iio: adc: mcp3911: simplify usage of spi->dev
14  [PATCH v8 4/6] iio: adc: mcp3911: fix indentation
15  [PATCH v8 5/6] iio: adc: mcp3911: avoid ambiguity parameters in macros
16  [PATCH v8 6/6] iio: adc: mcp3911: add support for the whole MCP39xx family
17  ---
18  NOTE: install dkimpy for DKIM signature verification
19---
20Total patches: 6
21---
22 Base: using specified base-commit b320441c04c9bea76cbee1196ae55c20288fd7a6
23Applying: dt-bindings: iio: adc: mcp3911: add support for the whole MCP39xx family
24Applying: iio: adc: mcp3911: make use of dev_err_probe()
25Applying: iio: adc: mcp3911: simplify usage of spi->dev
26Applying: iio: adc: mcp3911: fix indentation
27Applying: iio: adc: mcp3911: avoid ambiguity parameters in macros
28Applying: iio: adc: mcp3911: add support for the whole MCP39xx family

Or even b4 prep to create a branch for it. This will not fetch the latest version though.

 1$ b4 prep -n review -F  20230820102610.755188-6-marcus.folkesson@gmail.com
 2Checking attestation on all messages, may take a moment...
 3---
 4  [PATCH v7 1/6] dt-bindings: iio: adc: mcp3911: add support for the whole MCP39xx family
 5  [PATCH v7 2/6] iio: adc: mcp3911: make use of dev_err_probe()
 6  [PATCH v7 3/6] iio: adc: mcp3911: simplify usage of spi->dev
 7  [PATCH v7 4/6] iio: adc: mcp3911: fix indentation
 8  [PATCH v7 5/6] iio: adc: mcp3911: avoid ambiguity parameters in macros
 9  [PATCH v7 6/6] iio: adc: mcp3911: add support for the whole MCP39xx family
10  ---
11  NOTE: install dkimpy for DKIM signature verification
12---
13Created new branch b4/review
14Applying 6 patches
15---
16Applying: dt-bindings: iio: adc: mcp3911: add support for the whole MCP39xx family
17Applying: iio: adc: mcp3911: make use of dev_err_probe()
18Applying: iio: adc: mcp3911: simplify usage of spi->dev
19Applying: iio: adc: mcp3911: fix indentation
20Applying: iio: adc: mcp3911: avoid ambiguity parameters in macros
21Applying: iio: adc: mcp3911: add support for the whole MCP39xx family

Once you have the patches applied to your local repository it's easier to perform a review as it gives a better context when you can jump around in the codebase. Also, it allows you to run any scripts for sanity checks and such.

That is pretty much how I use b4 in the review process. b4 has a lot of more neat features such as fetching pull-requests or generate thank-emails when something gets merged/applied, but that is nothing I use for the moment.

Contributor's workflow

As I said, I've been unaware of that b4 can assist you even in the workflow as a contributor. I'm so excited!

The workflow

These steps is more or less copied directly from the documentation [1].

  1. Prepare your patch series by using b4 prep and queueing your commits. Use git rebase -i to arrange the commits in the right order and to write good commit messages.
  2. Prepare your cover letter using b4 prep --edit-cover. You should provide a good overview of what your series does and why you think it will improve the current code.
  3. When you are almost ready to send, use b4 prep --auto-to-cc to collect the relevant addresses from your commits. If your project uses a MAINTAINERS file, this will also perform the required query to figure out who should be included on your patch series submission.
  4. Review the list of addresses that were added to the cover letter and, if you know what you're doing, remove any that you think are unnecessary.
  5. Send your series using b4 send. This will automatically reroll your series to the next version and add changelog entries to the cover letter.
  6. Await code review and feedback from maintainers.
  7. Apply any received code-review trailers using b4 trailers -u.
  8. Use git rebase -i to make any changes to the code based on the feedback you receive. Remember to record these changes in the cover letter's changelog.
  9. Unless series is accepted upstream, GOTO 3.
  10. Clean up obsolete prep-managed branches using b4 prep --cleanup

Example of usage

A Lego loving friend of mine pointed out that a reference in kernel documentation [6] I wrote no longer points to the site it was supposed to. That is something we are going to fix!

Just follow the steps listed above.

Start by prepare the tree with b4 prep -n pxrc -f v6.3:

1$ b4 prep -n pxrc -f v6.3
2Created new branch b4/pxrc
3Created the default cover letter, you can edit with --edit-cover.

Now we have a branch for our work. We base our work on the v6.3 tag.

Next step is to edit the coverletter by b4 prep --edit-cover.

Here is what the first patch (coverletter) looks like after editing:

 1$ git show HEAD
 2commit b97650b087d88d113b11cd1bc367c67c314d77a1 (HEAD -> b4/pxrc)
 3Author: Marcus Folkesson <marcus.folkesson@gmail.com>
 4Date:   Fri Aug 25 11:43:52 2023 +0200
 5
 6    Remove reference to site that is no longer related
 7    
 8    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
 9    
10    --- b4-submit-tracking ---
11    # This section is used internally by b4 prep for tracking purposes.
12    {
13      "series": {
14        "revision": 1,
15        "change-id": "20230825-pxrc-8518b297cd21",
16        "prefixes": []
17      }
18    }

Notice the meta information about this series (revision, change-id). The change-id will follow us through all the versions of the patch.

Next is to commit the changes as usual with git add and git commit:

 1$ git add Documentation/input/devices/pxrc.rst
 2$ git commit --signoff
 3$ git show
 4commit 00cf9f943529c06b36e89407aecc18d46f1b028e (HEAD -> b4/pxrc)
 5Author: Marcus Folkesson <marcus.folkesson@gmail.com>
 6Date:   Thu Aug 24 15:47:24 2023 +0200
 7
 8    input: docs: pxrc: remove reference to phoenix-sim
 9    
10    The reference undeniably points to something unrelated nowadays.
11    Remove it.
12    
13    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
14
15diff --git a/Documentation/input/devices/pxrc.rst b/Documentation/input/devices/pxrc.rst
16index ca11f646bae8..5a86df4ad079 100644
17--- a/Documentation/input/devices/pxrc.rst
18+++ b/Documentation/input/devices/pxrc.rst
19@@ -5,7 +5,7 @@ pxrc - PhoenixRC Flight Controller Adapter
20 :Author: Marcus Folkesson <marcus.folkesson@gmail.com>
21 
22 This driver let you use your own RC controller plugged into the
23-adapter that comes with PhoenixRC [1]_ or other compatible adapters.
24+adapter that comes with PhoenixRC or other compatible adapters.
25 
26 The adapter supports 7 analog channels and 1 digital input switch.
27 
28@@ -41,7 +41,7 @@ Manual Testing
29 ==============
30 
31 To test this driver's functionality you may use `input-event` which is part of
32-the `input layer utilities` suite [2]_.
33+the `input layer utilities` suite [1]_.
34 
35 For example::
36 
37@@ -53,5 +53,4 @@ To print all input events from input `devnr`.
38 References
39 ==========
40 
41-.. [1] http://www.phoenix-sim.com/
42-.. [2] https://www.kraxel.org/cgit/input/
43+.. [1] https://www.kraxel.org/cgit/input/

Verify that the patch looks good with scripts/checkpatch.pl:

1$ ./scripts/checkpatch.pl --git HEAD
2total: 0 errors, 0 warnings, 22 lines checked
3
4Commit 00cf9f943529 ("input: docs: pxrc: remove reference to phoenix-sim") has no obvious style problems and is ready for submission.

Ok, the patch is ready to be sent out to the mailing list.

Collect all TO and CC from scripts/get_maintainer.pl with b4 prep --auto-to-cc:

 1$ b4 prep --auto-to-cc
 2Will collect To: addresses using get_maintainer.pl
 3Will collect Cc: addresses using get_maintainer.pl
 4Collecting To/Cc addresses
 5    + To: Dmitry Torokhov <dmitry.torokhov@gmail.com>
 6    + To: Jonathan Corbet <corbet@lwn.net>
 7    + Cc: linux-input@vger.kernel.org
 8    + Cc: linux-doc@vger.kernel.org
 9    + Cc: linux-kernel@vger.kernel.org
10---
11You can trim/expand this list with: b4 prep --edit-cover
12Invoking git-filter-repo to update the cover letter.
13New history written in 0.02 seconds...
14Completely finished after 0.18 seconds.

Now we are ready to send the patch to mailing list by invoking the b4 send command.

b4 send will automatically use the [sendemail] section of your git config to determine which SMTP server to use.

I've configure git to use gmail as SMTP server, here is the relevant part of my ~/.gitconfig:

[sendemail]
  smtpserver = smtp.gmail.com
  smtpuser = marcus.folkesson@gmail.com
  smtpserverport = 587
  smtpencryption = tls
  smtpssl = true
        chainreplyto = false
        confirm = auto

b4 send will send the patches to the mailing list and prepare for version 2 of the patch series by increase the version number and create a new tag.

Other features

Compare between versions

As I only have one version of the patch (and there will probably not be a version 2), I have no use for the cool compare feature. However, b4 let you compare your versions by simply :

b4 prep --compare-to v1

Trailers

Going through all mail threads and collect all trailer tags could be a painstaking work. B4 will do this for you and put the tag into the right patch magically.

Unfortunately, I forgot to mention my friend in the original patch, so I sent another mail [7] with the Suggested-by: Mark Olsson <mark@markolsson.se> tag.

Fetch the tag with b4 trailers -S -u (-S because the tag was sent by me and not Mark):

 1$ b4 trailers -S -u
 2Calculating patch-ids from commits, this may take a moment...
 3Checking change-id "20230824-pxrc-doc-1addbaa2250f"
 4Grabbing search results from lore.kernel.org
 5---
 6  input: docs: pxrc: remove reference to phoenix-sim
 7    + Suggested-by: Mark Olsson <mark@markolsson.se>
 8---
 9Invoking git-filter-repo to update trailers.
10New history written in 0.03 seconds...
11Completely finished after 0.17 seconds.
12Trailers updated.

As we can see the Suggested-by tag is now applied to the patch in the right place:

 1commit 650264be66f0e589cf67a49f769ddc7d51e076cb (HEAD -> b4/pxrc)
 2Author: Marcus Folkesson <marcus.folkesson@gmail.com>
 3Date:   Thu Aug 24 15:47:24 2023 +0200
 4
 5    input: docs: pxrc: remove reference to phoenix-sim
 6    
 7    The reference undeniably points to something unrelated nowadays.
 8    Remove it.
 9    
10    Suggested-by: Mark Olsson <mark@markolsson.se>
11    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Magic.

Further reading

The creator of the tool, Konstantin Ryabitsev, has an excellent youtube [8] video where he demonstrate the usage of b4.

CATALOG
    FEATURED TAGS
    audio beer birds blacksmith booting buildroot camera clockwork cultivation debugging device drivers dnd docker git go ham home control hugo knives leatherwork libostree linux linux kernel network programming qemu radio swupdate systemd til tools v4l2 vim wireless yocto zephyr