How does pathlib combine paths using slashes?
— 1 min read
Pathlib is a python module for working with filesystem paths that's part of the standard library. If you look at the official python docs for how to use pathlib, one of the first examples is this interesting snippet showing how to navigate inside a directory tree:
1>>> p = Path('/etc')2>>> q = p / 'init.d' / 'reboot'3>>> q4PosixPath('/etc/init.d/reboot')5>>> q.resolve()6PosixPath('/etc/rc.d/init.d/halt')
How does this work? How does pathlib use the forward-slash character to construct paths?
Magic Methods
To understand how this works, we need to first have a basic knowledge of magic methods, otherwise called dunder (which stands for double underscore) methods. If you've used python for a while, you've probably written such a method without necessarily realizing it. When you create a class, you typically define a function named __init__
. This magic method is automatically called whenever a new instance of your class is initialized. The same is true for other magic methods - they're not meant to be called directly by you, but are automatically called when you interact with an object.
Another example would be when you call str()
on an object - automatically, python looks at the class that's passed into the str
function to see if a __str__
method is defined inside that class. If so, that method will be executed and the result will be returned.
This is how the basic operators are defined in python as well. When we call 3 + 4
, python implicitly calls 3.__add__(4)
which looks up the __add__
method inside the int
class to know what should be returned. And, as you might've guessed, there's also a magic method associated with the /
operator called __truediv__
. That's exactly the method that's implemented inside the pathlib
module, as we can see by looking at the Python source code:
1# pathlib.py2def __truediv__(self, key):3 try:4 return self._make_child((key,))5 except TypeError:6 return NotImplemented7
8def _make_child(self, args):9 drv, root, parts = self._parse_args(args)10 drv, root, parts = self._flavour.join_parsed_parts(11 self._drv, self._root, self._parts, drv, root, parts)12 return self._from_parsed_parts(drv, root, parts)
Implementing the /
operator ourselves
Now that we understand how pathlib implements the /
operator, let's try to implement a simple example ourselves. First, we create a simple class with an enum
:
1from enum import Enum2
3
4class Color(Enum):5 BLUE = 16 GREEN = 117 YELLOW = 108 ORANGE = 1109 RED = 10010 PURPLE = 10111 BLACK = 111
If we try to use the /
operator with our class without implementing the __truediv__
magic method first, we see the following error message:
1>>> my_color = Color.BLUE2>>> your_color = Color.YELLOW3>>> my_color / your_color4Traceback (most recent call last):5 File "<stdin>", line 1, in <module>6TypeError: unsupported operand type(s) for /: 'Color' and 'Color
Let's now add an implementation for __truediv__
to our class:
1from enum import Enum2
3
4class Color(Enum):5 BLUE = 16 GREEN = 117 YELLOW = 108 ORANGE = 1109 RED = 10010 PURPLE = 10111 BLACK = 11112
13 def __truediv__(self, other):14 try:15 mixed_value = self.value + other.value16 return Color(mixed_value)17 except ValueError:18 return Color.BLACK19 # Important to catch the `TypeError` in case someone20 # tries to use `/` with our class and some other type21 except TypeError:22 return NotImplemented
If we try again to use the /
operator with instances of our class, we see the following output:
1>>> my_color = Color.BLUE2>>> your_color = Color.YELLOW3>>> my_color / your_color4<Color.GREEN: 11>5my_color / your_color / Color.RED6<Color.BLACK: 111>
You now know how to implement any operator in python, including ones like @
, ~
, %=
, <<
and >>
. Incidentally, these last two are heavily used by Apache Airflow, a popular workflow orchestration tool. For a full list of all the magic methods you can use, check out the python documentation for Data model.