How does pathlib combine paths using slashes?
Posted on July 23, 2021 • 3 minutes • 634 words
Pathlib is a python module for working with filesystem paths that’s part of the standard library. If you look at the official python docs for how to use pathlib, one of the first examples is this interesting snippet showing how to navigate inside a directory tree:
>>> p = Path('/etc') >>> q = p / 'init.d' / 'reboot' >>> q PosixPath('/etc/init.d/reboot') >>> q.resolve() PosixPath('/etc/rc.d/init.d/halt')
How does this work? How does pathlib use the forward-slash character to construct paths?
To understand how this works, we need to first have a basic knowledge of magic methods, otherwise called dunder (which stands for double underscore) methods. If you’ve used python for a while, you’ve probably written such a method without necessarily realizing it. When you create a class, you typically define a function named
__init__. This magic method is automatically called whenever a new instance of your class is initialized. The same is true for other magic methods - they’re not meant to be called directly by you, but are automatically called when you interact with an object.
Another example would be when you call
str() on an object - automatically, python looks at the class that’s passed into the
str function to see if a
__str__ method is defined inside that class. If so, that method will be executed and the result will be returned.
This is how the basic operators are defined in python as well. When we call
3 + 4, python implicitly calls
3.__add__(4) which looks up the
int class to know what should be returned. And, as you might’ve guessed, there’s also a magic method associated with the
/ operator called
__truediv__ that’s listed in the docs
. That’s exactly the method that’s implemented inside the
pathlib module, as we can see by looking at the Python source code
# pathlib.py def __truediv__(self, key): try: return self._make_child((key,)) except TypeError: return NotImplemented def _make_child(self, args): drv, root, parts = self._parse_args(args) drv, root, parts = self._flavour.join_parsed_parts( self._drv, self._root, self._parts, drv, root, parts) return self._from_parsed_parts(drv, root, parts)
/ operator ourselves
Now that we understand how pathlib implements the
/ operator, let’s try to implement a simple example ourselves. First, we create a simple class with an
from enum import Enum class Color(Enum): BLUE = 1 GREEN = 11 YELLOW = 10 ORANGE = 110 RED = 100 PURPLE = 101 BLACK = 111
If we try to use the
/ operator with our class without implementing the
__truediv__ magic method first, we see the following error message:
>>> my_color = Color.BLUE >>> your_color = Color.YELLOW >>> my_color / your_color Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for /: 'Color' and 'Color
Let’s now add an implementation for
__truediv__ to our class:
from enum import Enum class Color(Enum): BLUE = 1 GREEN = 11 YELLOW = 10 ORANGE = 110 RED = 100 PURPLE = 101 BLACK = 111 def __truediv__(self, other): try: mixed_value = self.value + other.value return Color(mixed_value) except ValueError: return Color.BLACK # Important to catch the `TypeError` in case someone # tries to use `/` with our class and some other type except TypeError: return NotImplemented
If we try again to use the
/ operator with instances of our class, we see the following output:
>>> my_color = Color.BLUE >>> your_color = Color.YELLOW >>> my_color / your_color <Color.GREEN: 11> my_color / your_color / Color.RED <Color.BLACK: 111>
You now know how to implement any operator in python, including ones like @ , ~ , %= , « and ». Incidentally, these last two are heavily used by Apache Airflow , a popular workflow orchestration tool. For a full list of all the magic methods you can use, check out the python documentation for Data model .