Partial Python code formatting with Black & PyCharm

In the Python world there are several code formatters - e.g. Black, YAPF and autopep8. My personal preference is Black as it is deliberately unconfigurable; there's not much to configure and the tool is rather opinionated about formatting code, resulting in me sometimes hitting ⌥⌘L in PyCharm and Black doing the rest. Notice: this blog post is written specifically with PyCharm and MacOS, although the ideas can be used elsewhere.

For example, Black will automatically format this:

mything = DemoClass('hello', "world",
                    arg1="this",
                        arg2="is",
                            arg3='very',
                                arg4="ugly",
                                    arg5="formatting",
                    arg6="nobody codes like this anyways")

Into this:

mything = DemoClass(
    "hello",
    "world",
    arg1="this",
    arg2="is",
    arg3="very",
    arg4="ugly",
    arg5="formatting",
    arg6="nobody codes like this anyways",
)

However, this poses an issue when working on open source projects as the code reformatting is applied on the entire file, including code that I don't want to touch. This causes me to reformat the code, copy the changed lines I wanted to format, revert formatting, and replace my lines by the copied lines. This felt clumsy and so I went to investigate.

Installation and configuration in PyCharm

First, we need to install Black and configure it in PyCharm. It's available via both Pip and Conda, and runs on Python 3.6 and higher, although it can format older Python code too. After installation you can run black /path/to/your/file to format it. Note the exact Black binary location; for example with Conda, the binary is (default) located in /Users/[username]/miniconda3/envs/[envname]/bin/black.

Black PyCharm configuration

To use Black in PyCharm, go to PyCharm -> Preferences... (⌘,) -> Tools -> External Tools -> Click + symbol to add new external tool. Configure as shown above and to reformat your current file, go to Tools -> External Tools -> Black. Additionally you could override PyCharm's default Reformat Code shortcut with Black by configuring a keymap.

Partial formatting

Black only formats entire files so this poses an issue when working on open source code. Luckily, in the PyCharm external tool configuration there are many variables available. In the configuration, click on Insert Macro... to see them all:

PyCharm external tool macros

To apply partial Black formatting, we need at least the selected line numbers which are given by SelectionStartLine and SelectionEndLine. I created a small Bash script to call as external tool in PyCharm. Note the sed commands are MacOS specific:

#!/usr/bin/env bash

set -x

black=$1
input_file=$2
start_line=$3
end_line=$4

# Read selected lines and write to tmpfile
selection=$(sed -n "$start_line, $end_line p; $(($end_line+1)) q" < $input_file)
tmpfile=$(mktemp)
echo "$selection" > "$tmpfile"

# Apply Black formatting to tmpfile
$black $tmpfile

# Delete original lines from file
sed -i "" "$start_line,$end_line d" $input_file

# And insert newly formatted lines
sed -i "" "$(($start_line-1)) r $tmpfile" $input_file

Code is also available at https://gdd.li/black-selection.

Let's break it down:

1. PyCharm configuration

The script accepts four arguments: (1) Black location, (2) input file, (3) selection start line and (4) selection end line.

black=$1
input_file=$2
start_line=$3
end_line=$4

The script is called together with the four arguments by PyCharm:

Black PyCharm configuration for selection

2. Fetch selection

# Read selected lines and write to tmpfile
selection=$(sed -n "$start_line, $end_line p; $(($end_line+1)) q" < $input_file)
tmpfile=$(mktemp)
echo "$selection" > "$tmpfile"

With some sed magic, I fetch the selection from the input file and write that to a tmpfile. The sed command executes e.g. sed -n "4, 7 p; 8 q" < /path/to/input/file where 4 is the start line number, 7 the end line number, p to print the lines and 8 q to stop scanning the file after the following line, which is useful in case of very long files. The -n flag suppresses echoing to stdout. The result is the selected lines written to the tmpfile.

3. Apply Black on selection

# Apply Black formatting to tmpfile
$black $tmpfile

The Black binary is executed on the tmpfile.

4. Place formatted selection back into original file

# Delete original lines from file
sed -i "" "$start_line,$end_line d" $input_file

# And insert newly formatted lines
sed -i "" "$(($start_line-1)) r $tmpfile" $input_file

Sed on MacOS and newlines do not work together nicely and the sed command required for replacing lines by a variable containing newlines became too cumbersome. So I decided to write less magic and split the replacing of the result into two lines: (1) removing the original lines from the input file and (2) inserting the lines from the formatted tmpfile back into the input file. Sed on MacOS requires an extension for backups, which is given by the -i flag. Since I don't want backups, an empty string is given.

5. PyCharm shortcut

Finally, create a PyCharm shortcut for formatting a selection:

PyCharm keymap for formatting selection

With this ⌥⌘; shortcut we can now call the script to format only the selection:

PyCharm code selection formatted

Final words

The script is not fool-proof, e.g. if you select half a block of code, Black will fail to format. However it does make formatting code for open source projects just a little more efficient which makes me a happy programmer.

Stay up to date on the latest insights and best-practices by registering for the GoDataDriven newsletter.
Follow us for more of this