Linters and formatters¶
PEP8¶
PEP8 is the official style guide for python. It is important to know the style-guide if you want to be a part of the python-community.
- PEP8 coding conventions are:
- Spaces are the preferred indentation method.
- Use 4 spaces per indentation level.
- Limit all lines to a maximum of 79 characters.
- Separate top-level function and class definitions with two blank lines.
- Method definitions inside a class are surrounded by a single blank line.
- Imports should be grouped in the following order:
- Standard library imports.
- Related third party imports.
- Local application/library specific imports.
- A blank line between each group of imports.
pycodestyle¶
Pycodestyle (Formerly PEP8) is the official linter tool to check the python code against the style conventions of PEP8 python.
To install it:
pip install pycodestyle
.
Let us take a small example script to test pycodestyle
We will create a test script file test_script.py
and use it as an example for all the linters.
from __future__ import print_function
import os, sys
import logging
from .. import views
class DoSomething(SomeCommand) :
def __init__(self):
for i in range(1,11):
if self.number == i:
print("matched")
else:
print('not matched')
def check_user(self):
if self.user: return True
else : return False
If we run pycodestyle:
$ pycodestyle {source_file_or_directory}
$ pycodestyle test_script.py
test_script.py:2:10: E401 multiple imports on one line
test_script.py:6:1: E302 expected 2 blank lines, found 1
test_script.py:6:31: E203 whitespace before ':'
test_script.py:9:25: E231 missing whitespace after ','
test_script.py:13:37: W291 trailing whitespace
test_script.py:16:21: E701 multiple statements on one line (colon)
test_script.py:16:34: W291 trailing whitespace
test_script.py:17:13: E271 multiple spaces after keyword
test_script.py:17:14: E203 whitespace before ':'
test_script.py:17:15: E701 multiple statements on one line (colon)
test_script.py:17:29: W291 trailing whitespace
To see the summary, use --statistics -qq
$ pycodestyle --statistics -qq {source_file_or_directory}
$ pycodestyle --statistics -qq test_script.py
2 E203 whitespace before ':'
1 E231 missing whitespace after ','
1 E271 multiple spaces after keyword
1 E302 expected 2 blank lines, found 1
1 E401 multiple imports on one line
2 E701 multiple statements on one line (colon)
3 W291 trailing whitespace
We can also make pycodestyle show the error and the description of how to solve the error by using --show-source --show-pep8
$ pycodestyle --show-source --show-pep8 {source_file_or_directory}
$ pycodestyle --show-source --show-pep8 test_script.py
test_script.py:2:10: E401 multiple imports on one line
import os, sys
^
Place imports on separate lines.
...
...
...
To customise pycodestyle we can configure it at the project-level or in user-level. It is better to configure at the project level as the style usually varies for every-project.
To configure a project’s pycodestyle create a tox.ini
Or a setup.cfg
And add
[pycodestyle]
ignore = E501, W291
max-line-length = 88
statistics = True
- In the above file ,
[pycodestyle]
tells this is the pycodestyle section- we are telling to ignore the Error E501 (which is a line-length error) and Warning W291 (trailing whitespace warning).
- mentioning the max-line-length to be 88.
- and to show the statistics with every check
Error/ Warning | Meaning |
---|---|
Starting with E… | Errors |
Starting with W… | Warnings |
100 type … | Indentation |
200 type … | Whitespace |
300 type … | Blank lines |
400 type … | Imports |
500 type … | Line length |
600 type … | Deprecation |
700 type … | Statements |
900 type … | Syntax errors |
pylint¶
Pylint is a python linter which checks the source code and also acts as a bug and quality checker. It has more verification checks and options than just PEP8(Python style guide).
This is the most commonly used tool for linting in python.
- It includes the following features:
- Checking the length of each line
- Checking if variable names are well-formed according to the project’s coding standard
- Checking if declared interfaces are truly implemented.
To install it:
pip install pylint
.
Usage:
pylint {source_file_or_directory}
Using the file test_script.py as an example
$ pylint test_script.py
No config file found, using default configuration
************* Module test_script
C: 6, 0: No space allowed before :
class DoSomething(SomeCommand) :
^ (bad-whitespace)
C: 9, 0: Exactly one space required after comma
for i in range(1,11):
^ (bad-whitespace)
C: 13, 0: Trailing whitespace (trailing-whitespace)
C: 16, 0: Trailing whitespace (trailing-whitespace)
C: 17, 0: Final newline missing (missing-final-newline)
C: 17, 0: No space allowed before :
else : return False
^ (bad-whitespace)
C: 1, 0: Missing module docstring (missing-docstring)
C: 2, 0: Multiple imports on one line (os, sys) (multiple-imports)
E: 4, 0: Attempted relative import beyond top-level package (relative-beyond-top-level)
C: 6, 0: Missing class docstring (missing-docstring)
E: 6,18: Undefined variable 'SomeCommand' (undefined-variable)
C: 15, 4: Missing method docstring (missing-docstring)
R: 16, 8: The if statement can be replaced with 'return bool(test)' (simplifiable-if-statement)
R: 16, 8: Unnecessary "else" after "return" (no-else-return)
C: 16,22: More than one statement on a single line (multiple-statements)
R: 6, 0: Too few public methods (1/2) (too-few-public-methods)
W: 2, 0: Unused import sys (unused-import)
W: 2, 0: Unused import os (unused-import)
W: 3, 0: Unused import logging (unused-import)
W: 4, 0: Unused import views (unused-import)
----------------------------------------------------------------------
Your code has been rated at -10.00/10 (previous run: -10.00/10, +0.00)
As we can see pylint has more error/warning checks and options than pep8. And it is more descriptive.
To customise pylint we can configure it at the project-level, user-level or global-level .
- create a
/etc/pylintrc
for default global configuration - create a
~/pylintrc
for default user configuration - Or create a
pylintrc
file
To create a pylintrc file pylint --generate-rcfile > pylintrc
, which creates a template pylintrc(with comments) which can be customised as required.
For example if we want the max line lenght to be 88, then we have to set the max-line-length
to 88 .
pyflakes¶
pyflakes is a verification tool(linter) which checks for Python files for errors. Pyflakes doesn’t verify the style at all but it verifies only logistic errors like the syntax tree of each file individually.
To install it:
pip install pyflakes
.
Let us take the same example script to test pyflakes
Usage:
pyflakes {source_file_or_directory}
Using the file test_script.py as an example
$ pyflakes test_script.py
test_script.py:2: 'sys' imported but unused
test_script.py:2: 'os' imported but unused
test_script.py:3: 'logging' imported but unused
test_script.py:4: '..views' imported but unused
test_script.py:6: undefined name 'SomeCommand'
It detected newly “library imported but unused” and “Undefined name”, it doesn’t verify the style but verify only logistic error.
If we like Pyflakes but also want stylistic checks, we can use flake8, which combines Pyflakes with style checks against PEP 8
flake8¶
Flake8 is just a wrapper around pyflakes, pycodestyle and McCabe script (circular complexity checker) (which is used to detect complex-code).
If we like Pyflakes but also want stylistic checks, we can use flake8, which combines Pyflakes with style checks against PEP 8
To install it:
pip install flake8
.
Usage:
flake8 {source_file_or_directory}
To get statics also
flake8 {source_file_or_directory} --statistics
Using the file test_script.py as an example
$ flake8 test_script.py --statistics
test_script.py:2:1: F401 'os' imported but unused
test_script.py:2:1: F401 'sys' imported but unused
test_script.py:2:10: E401 multiple imports on one line
test_script.py:3:1: F401 'logging' imported but unused
test_script.py:4:1: F401 '..views' imported but unused
test_script.py:6:1: E302 expected 2 blank lines, found 1
test_script.py:6:19: F821 undefined name 'SomeCommand'
test_script.py:6:31: E203 whitespace before ':'
test_script.py:9:25: E231 missing whitespace after ','test_script.py:13:37: W291 trailing whitespace
test_script.py:16:21: E701 multiple statements on one line (colon)
test_script.py:16:34: W291 trailing whitespace
test_script.py:17:13: E271 multiple spaces after keyword
test_script.py:17:14: E203 whitespace before ':'
test_script.py:17:15: E701 multiple statements on one line (colon)test_script.py:17:29: W291 trailing whitespace
2 E203 whitespace before ':'
1 E231 missing whitespace after ','
1 E271 multiple spaces after keyword
1 E302 expected 2 blank lines, found 1
1 E401 multiple imports on one line
2 E701 multiple statements on one line (colon)
4 F401 'os' imported but unused
1 F821 undefined name 'SomeCommand'
3 W291 trailing whitespace
The output is formatted as:
file path : line number : column number : error code : short description
By adding the --show-source
option, it’ll be easier to find out what parts of the source code need to be revised.
$ flake8 test_script.py --show-source
test_script.py:2:1: F401 'os' imported but unused
import os, sys
^
test_script.py:2:1: F401 'sys' imported but unused
import os, sys
^
test_script.py:2:10: E401 multiple imports on one line
import os, sys
^
test_script.py:3:1: F401 'logging' imported but unused
import logging
^
...
...
...
We can see the result of pep8 (error code is Exxx and Wxxx) and pyflakes (error code is Fxxx) are output together.
Flake8 Error code meaning
The error code of flake8 are :
- E***/W***: Errors and warnings of pep8
- F***: Detections of PyFlakes
- C9**: Detections of circulate complexity by McCabe-script
Flake8 can be customised/configured in :
- Toplevel User directory, in
~/.config/flake8
Or - In a project folder by one of
setup.cfg
,tox.ini
, or.flake8
.
To customize flake8
[flake8]
ignore = D203
exclude = .git,__pycache__,docs/source/conf.py,old,build,dist, *migrations*
max-complexity = 10
This is same as the below one line command
$ flake8 --ignore D203 \
--exclude .git,__pycache__,docs/source/conf.py,old,build,dist \
--max-complexity 10
black¶
black is a python code auto-formatter. Black reformats entire files in place and also formats the strings to have double-qoutes.
Black is not configurable(except for line-length).
To install it:
pip install black
.
Usage:
black {source_file_or_directory}
The response we got when we did black test_script.py
is
Using the file test_script.py as an example
And the formatted code is
from __future__ import print_function
import os, sys
import logging
from .. import views
class DoSomething(SomeCommand):
def __init__(self):
for i in range(1, 11):
if self.number == i:
print("matched")
else:
print("not matched")
def check_user(self):
if self.user:
return True
else:
return False
To customise black we have to add this section in pyproject.toml
[tool.black]
line-length = 90
py36 = true
include = '\.pyi?$'
exclude = '''
/(
\.git
| \.mypy_cache
| \.tox
| \.venv
| _build
| buck-out
| build
| dist
)/
'''
In the above section, we have modified the line-lenght to 90, and specified the python version to 3.6
autopep8¶
autopep8 automatically formats Python code to the PEP8 style. It fixes most of the formatting issues that are reported by pycodestyle.
To install it:
pip install autopep8
Usage(to format a file):
autopep8 --in-place {file_name}
here --in-place
is to make changes to files in place.
Using the file test_script.py as an example
This is the formatted code.
from __future__ import print_function
import os
import sys
import logging
from .. import views
class DoSomething(SomeCommand):
def __init__(self):
for i in range(1, 11):
if self.number == i:
print("matched")
else:
print('not matched')
def check_user(self):
if self.user:
return True
else:
return False
To configure autopep8 we have to add this section [pep8]
in setup.cfg
:
[pep8]
ignore = E226,E302,E41
max-line-length = 160
yapf¶
Yet another Python formatter is another auto-formatter which is maintained by google. yapf is highly configurable and it has different base configurations, like pep8, Google and Facebook.
To install it:
pip install yapf
Usage:
yapf -i {source_file_or_directory}
here -i
is to make changes to files in place.
This is the formatted code.
from __future__ import print_function
import os, sys
import logging
from .. import views
class DoSomething(SomeCommand):
def __init__(self):
for i in range(1, 11):
if self.number == i:
print("matched")
else:
print('not matched')
def check_user(self):
if self.user: return True
else: return False
To configure yapf we have to add this section [yapf]
in setup.cfg
:
[yapf]
ignore = E226,E302
max-line-length = 96
Conclusion¶
Linting:
Pylint and flake8 have the most detailed way of showing the error and warnings(and it also gives the code rating).