Partial Python code formatting with Black & PyCharm
In the Python world there are several code formatters - e.g. Black, YAPF and autopep8. My personal preference is Black as it is deliberately unconfigurable; there's not much to configure and the tool is rather opinionated about formatting code, resulting in me sometimes hitting
⌥⌘L in PyCharm and Black doing the rest. Notice: this blog post is written specifically with PyCharm and MacOS, although the ideas can be used elsewhere.
For example, Black will automatically format this:
mything = DemoClass('hello', "world", arg1="this", arg2="is", arg3='very', arg4="ugly", arg5="formatting", arg6="nobody codes like this anyways")
mything = DemoClass( "hello", "world", arg1="this", arg2="is", arg3="very", arg4="ugly", arg5="formatting", arg6="nobody codes like this anyways", )
However, this poses an issue when working on open source projects as the code reformatting is applied on the entire file, including code that I don't want to touch. This causes me to reformat the code, copy the changed lines I wanted to format, revert formatting, and replace my lines by the copied lines. This felt clumsy and so I went to investigate.
Installation and configuration in PyCharm
First, we need to install Black and configure it in PyCharm. It's available via both Pip and Conda, and runs on Python 3.6 and higher, although it can format older Python code too. After installation you can run
black /path/to/your/file to format it. Note the exact Black binary location; for example with Conda, the binary is (default) located in /Users/[username]/miniconda3/envs/[envname]/bin/black.
To use Black in PyCharm, go to PyCharm -> Preferences... (⌘,) -> Tools -> External Tools -> Click
+ symbol to add new external tool. Configure as shown above and to reformat your current file, go to Tools -> External Tools -> Black. Additionally you could override PyCharm's default Reformat Code shortcut with Black by configuring a keymap.
Black only formats entire files so this poses an issue when working on open source code. Luckily, in the PyCharm external tool configuration there are many variables available. In the configuration, click on
Insert Macro... to see them all:
To apply partial Black formatting, we need at least the selected line numbers which are given by
SelectionEndLine. I created a small Bash script to call as external tool in PyCharm. Note the sed commands are MacOS specific:
#!/usr/bin/env bash set -x black=$1 input_file=$2 start_line=$3 end_line=$4 # Read selected lines and write to tmpfile selection=$(sed -n "$start_line, $end_line p; $(($end_line+1)) q" < $input_file) tmpfile=$(mktemp) echo "$selection" > "$tmpfile" # Apply Black formatting to tmpfile $black $tmpfile # Delete original lines from file sed -i "" "$start_line,$end_line d" $input_file # And insert newly formatted lines sed -i "" "$(($start_line-1)) r $tmpfile" $input_file
Code is also available at https://gdd.li/black-selection.
Let's break it down:
1. PyCharm configuration
The script accepts four arguments: (1) Black location, (2) input file, (3) selection start line and (4) selection end line.
black=$1 input_file=$2 start_line=$3 end_line=$4
The script is called together with the four arguments by PyCharm:
2. Fetch selection
# Read selected lines and write to tmpfile selection=$(sed -n "$start_line, $end_line p; $(($end_line+1)) q" < $input_file) tmpfile=$(mktemp) echo "$selection" > "$tmpfile"
With some sed magic, I fetch the selection from the input file and write that to a tmpfile. The sed command executes e.g.
sed -n "4, 7 p; 8 q" < /path/to/input/file where 4 is the start line number, 7 the end line number,
p to print the lines and
8 q to stop scanning the file after the following line, which is useful in case of very long files. The
-n flag suppresses echoing to stdout. The result is the selected lines written to the tmpfile.
3. Apply Black on selection
# Apply Black formatting to tmpfile $black $tmpfile
The Black binary is executed on the tmpfile.
4. Place formatted selection back into original file
# Delete original lines from file sed -i "" "$start_line,$end_line d" $input_file # And insert newly formatted lines sed -i "" "$(($start_line-1)) r $tmpfile" $input_file
Sed on MacOS and newlines do not work together nicely and the sed command required for replacing lines by a variable containing newlines became too cumbersome. So I decided to write less magic and split the replacing of the result into two lines: (1) removing the original lines from the input file and (2) inserting the lines from the formatted tmpfile back into the input file. Sed on MacOS requires an extension for backups, which is given by the
-i flag. Since I don't want backups, an empty string is given.
5. PyCharm shortcut
Finally, create a PyCharm shortcut for formatting a selection:
⌥⌘; shortcut we can now call the script to format only the selection:
The script is not fool-proof, e.g. if you select half a block of code, Black will fail to format. However it does make formatting code for open source projects just a little more efficient which makes me a happy programmer.
Follow us for more of this
Highlights from the new Apache Avro 1.9.0 release
May 14, 2019
May 12, 2019
GoDataDriven Open Source Contribution for March and April 2019
May 03, 2019
Dutch Data Science Week 2019
April 25, 2019
Migrating your Hadoop workloads to the Cloud
April 22, 2019
GoDataDriven announces Data Council NL community
April 19, 2019