This week I set out to make some changes to my source code environment to support contributors to the project. The main goal was to add a formatter and a linter to the environment to support consistent code formatting and layout as other developers contribute code changes.
To do this I worked with Black Formatter and Flake8 Linter as they were both tools I saw used frequently and with good feedback from Python developers. Both were very simple to install and implement.
Black
Black was a straightforward install pip install black from CLI.
Once installed it can be run on a .py file or set of directories containing files with the command black .\src\. from my project base directory. This formats every file in src directory and every child directory. Including a filename instead of “.” works on just the targeted file. The changes to the files are implemented and saved automatically.
Flake8
Flake8 required the exact same process to install and a slightly different process to run. From CLI python -m pip install flake8 installs the extension into your local environment or any virtual environment active where the install command was executed (this applies to the Black installation above as well).
Once installed I ran flake* .\src\. to lint all .py files in my src directory and its child folders. The same path changes can be made to apply flake8 to just a single file.
Rather than automatically editing and saving your files, flaked8 run from CLI provides a list of suggested changes for each file targeted, including the line and column number of the issue and a message describing the suggested edit. This can be altered in your editor and flake8 should be run repeatedly until you have satisfied all the suggested edits. If there is a particular issue you would like ignored when running the application, you can run flake8 --extend-ignore <<Error Code>> <<path/to/files>> instead.
VSCode Integration
Both of these tools offer VSCode extensions to simplify the process. They can be searched via the VSCode extensions window by typing Ctrl + Shift + X and searching “Black Formatter” and “Flake8” in the extensions sidebar. Just install and/or enable the extension. For Black I chose to apply the Black formatting automatically on save, this can be setup in VSCode Settings Ctrl + , and searching “format on save” – by selecting the format on save checkbox, Black automatically makes format changes to your .py files each time they are saved.
Conflicts
In addition to command line arguments that can be used to ignore or customize the format applied by both programs, you can also customize each application with settings in a pyproject.toml file.
I was actually running into a conflict between the default settings of Black and Flake8 that when used together actually had a disagreement as to what style to adhere to. Specifically the 2 tools enforced different defaults for maximum line length. So as Flake8 made a recommendation to shorten a line based on a smaller maximum line length setting, Black would immediately change the edits made upon saving the file.
To solve this I chose to reduce the line-length default in Black to match that of Flake8. I did this by simply adding the following code to my “pyproject.toml” file.
[tool.black]
line-length = 79
Results
The format changes that Black made to my code were minor although they did implement the consistency desired from its use.
The majority of the changes made by Black were:
- Changing all single quotes to double quotes.
- Adding an extra blank line between functions.
- Corrected a few instances of incorrect indentation.
Flake8 suggested a number of edits as well some of those included
- missing whitespace around operator
- blank line contains whitespace
- line too long (91 > 79 characters)
- continuation line under-indented for visual indent
- missing whitespace after ','OR ‘(‘
- expected 2 blank lines, found 1
- too many blank lines (2)
- f-string is missing placeholders
- no newline at end of file
While not consequential to the functionality of the code, the changes enforced by these 2 applications adds a great deal of conformity to all code, and can really ease the burden and concern one has when seeking outside contribution to your project. It takes a great deal of the guess work out of assessing and accepting edits made by others and certainly reduces the amount of potential conflicts presented by git when attempting to merge the contributions. This way everyone can focus more on the quality of code rather than style guide adherence.
While all of this was very straightforward to implement, I also wanted to simplify and streamline the setup of these tools for contributors setting up the project for the first time, as well as a single command option for running the tools against edited files.
At the end of the day this was an easy task that I tackled by creating stand alone python scripts that run a series of executables. For example, format.py was a simple way to run both Black and Flake8 on the entire source code.
While this ended up being the easy option, the truth is that it took me days diving down a best practices rabbit hole. Something about the prospect of preparing my project to facilitate ease of contribution drove me to get into some deep weeds about how to best setup python projects for open source from the initial stages. This fueled an exhausting dive into researching virtual environments, setuptools, packaging py projects, and various other approaches to building open source py projects and providing intuitive build processes that simplify setup, execution, and contribution process.
Being that I just pulled myself out of that rabbit hole and settled on simply providing some simple scripts to install dependencies and run formatters, I think the various take aways from this exhausting deep dive will have to wait for a future post, once I’ve had a little time to process and try out the many, varied methods out there.
Top comments (0)