How does pathlib combine paths using slashes?
— 1 min read
Pathlib is a python module for working with filesystem paths that's part of the standard library. If you look at the official python docs for how to use pathlib, one of the first examples is this interesting snippet showing how to navigate inside a directory tree:
1>>> p = Path('/etc')2>>> q = p / 'init.d' / 'reboot'3>>> q4PosixPath('/etc/init.d/reboot')5>>> q.resolve()6PosixPath('/etc/rc.d/init.d/halt')
How does this work? How does pathlib use the forward-slash character to construct paths?
To understand how this works, we need to first have a basic knowledge of magic methods, otherwise called dunder (which stands for double underscore) methods. If you've used python for a while, you've probably written such a method without necessarily realizing it. When you create a class, you typically define a function named
__init__. This magic method is automatically called whenever a new instance of your class is initialized. The same is true for other magic methods - they're not meant to be called directly by you, but are automatically called when you interact with an object.
Another example would be when you call
str() on an object - automatically, python looks at the class that's passed into the
str function to see if a
__str__ method is defined inside that class. If so, that method will be executed and the result will be returned.
This is how the basic operators are defined in python as well. When we call
3 + 4, python implicitly calls
3.__add__(4) which looks up the
__add__ method inside the
int class to know what should be returned. And, as you might've guessed, there's also a magic method associated with the
/ operator called
__truediv__. That's exactly the method that's implemented inside the
pathlib module, as we can see by looking at the Python source code:
1# pathlib.py2def __truediv__(self, key):3 try:4 return self._make_child((key,))5 except TypeError:6 return NotImplemented78def _make_child(self, args):9 drv, root, parts = self._parse_args(args)10 drv, root, parts = self._flavour.join_parsed_parts(11 self._drv, self._root, self._parts, drv, root, parts)12 return self._from_parsed_parts(drv, root, parts)
/ operator ourselves
Now that we understand how pathlib implements the
/ operator, let's try to implement a simple example ourselves. First, we create a simple class with an
1from enum import Enum234class Color(Enum):5 BLUE = 16 GREEN = 117 YELLOW = 108 ORANGE = 1109 RED = 10010 PURPLE = 10111 BLACK = 111
If we try to use the
/ operator with our class without implementing the
__truediv__ magic method first, we see the following error message:
1>>> my_color = Color.BLUE2>>> your_color = Color.YELLOW3>>> my_color / your_color4Traceback (most recent call last):5 File "<stdin>", line 1, in <module>6TypeError: unsupported operand type(s) for /: 'Color' and 'Color
Let's now add an implementation for
__truediv__ to our class:
1from enum import Enum234class Color(Enum):5 BLUE = 16 GREEN = 117 YELLOW = 108 ORANGE = 1109 RED = 10010 PURPLE = 10111 BLACK = 1111213 def __truediv__(self, other):14 try:15 mixed_value = self.value + other.value16 return Color(mixed_value)17 except ValueError:18 return Color.BLACK19 # Important to catch the `TypeError` in case someone20 # tries to use `/` with our class and some other type21 except TypeError:22 return NotImplemented
If we try again to use the
/ operator with instances of our class, we see the following output:
1>>> my_color = Color.BLUE2>>> your_color = Color.YELLOW3>>> my_color / your_color4<Color.GREEN: 11>5my_color / your_color / Color.RED6<Color.BLACK: 111>
You now know how to implement any operator in python, including ones like
>>. Incidentally, these last two are heavily used by Apache Airflow, a popular workflow orchestration tool. For a full list of all the magic methods you can use, check out the python documentation for Data model.