Introduction
Ever wondered why you can add two numbers with +
, check if something's in a list with in
, or use len()
on strings? The secret lies in Python's "magic methods" – special methods surrounded by double underscores (called "dunder" methods) that make your custom classes behave like built-in types.
“Dunder” methods, or magic methods in Python are one of the more interesting things you can use while programming.
They allow for overloading and extending core class functionalities, and are, at the core of Python, how basic operations work.
You may already be familiar with the class initializer dunder method: __init__
. This is a method that is automatically invoked whenever a class is initialized. It typically works similarly to a constructor
in c
based programming languages.
The magic behind operators
We can start with something that you use everyday, the +
operator. When you write something like "hello" + "world"
, Python is actually calling the __add__
method behind the scenes to overload the operator.
Here’s a simple example to demonstrate this:
class Temperature:
def __init__(self, celsius: int):
self.celsius = celsius
def __add__(self, other: object) -> object:
if isinstance(other, Temperature):
return Temperature(self.celsius + other.celsius)
def __str__(self) -> str:
return f"{self.celsius} C"
Now, we can add temperatures naturally!
t1 = Temperature(5)
t2 = Temperature(25)
t3 = t1 + t2 # this actually calls t1.__add__(t2)
print(t3) # 30 C
That works much more naturally then defining an add(other)
function! Try this yourself with __sub__
, the subtraction operator.
def __sub__(self, other: object) -> object:
if isinstance(other, Temperature):
return Temperature(self.celsius - other.celsius)
Notice how we have an isinstance
check? This prevents us from trying to add incompatible types together. If we were to do something like t1 + 3
, nothing would be returned. This is a problem. We can fix this in a couple of ways. The most Pythonic might be to convert the other value we are adding to a temperature, like so
def __add__(self, other: object) -> object:
if isinstance(other, Temperature):
return Temperature(self.celsius + other.celsius)
if isinstance(other, (int, float)):
return self + Temperature(other) # recursively calls __add__ again
However, this still does not handle other types! We need to return a NotImplemented
to inform Python, and the users of our class, that the attempted operation is not possible (we will discuss what Python tries when receiving this from an operation in the future.
def __add__(self, other: object) -> object:
if isinstance(other, Temperature):
return Temperature(self.celsius + other.celsius)
if isinstance(other, (int, float)):
return self + Temperature(other) # recursively calls __add__ again
return NotImplemented
Our first example
Let’s build small toy example to demonstrate some more of the arithmetic dunder methods.
class Money:
def __init__(self, amount: int | float, currency: str = "USD"):
self.amount = amount
self.currency = currency
def __add__(self, other: object) -> object:
if isinstance(other, (int, float)):
return self + Money(other, self.currency)
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(
f"Cannot add different currencies: {self.currency}, {other.currency}"
)
return Money(self.amount + other.amount)
return NotImplemented
def __sub__(self, other: object) -> object:
if isinstance(other, (int, float)):
return self - Money(other, self.currency)
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(
f"Cannot subtract different currencies: {self.currency}, {other.currency}"
)
return Money(self.amount - other.amount)
return NotImplemented
def __eq__(self, other: object) -> bool:
if isinstance(other, (int, float)):
return self == Money(other, self.currency)
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(
f"Cannot compare different currencies: {self.currency}, {other.currency}"
)
return self.amount == other.amount
return NotImplemented
def __lt__(self, other: object) -> bool:
if isinstance(other, (int, float)):
return self < Money(other, self.currency)
if isinstance(other, Money):
if self.currency != other.currency:
raise ValueError(
f"Cannot compare different currencies: {self.currency}, {other.currency}"
)
return self.amount < other.amount
return NotImplemented
def __str__(self):
return f"{self.amount:.2f} {self.currency}"
def __repr__(self):
return f"Money({self.amount}, '{self.currency}')"
Again, we can use these new methods naturally
price = Money(19.99)
tax = Money(2.00)
total = price + tax # Money(21.99, 'USD')
print(f"Total: {total}") # Total: 21.99
savings = Money(100.00)
print(total < savings) # True
print(repr(savings)) # Money(100.0, 'USD')
Let’s break down which methods we used here.
Basic Dunder Methods
__init__
As previously discussed, this functions essentially like a constructor (with a bit of difference, as there is also a __new__
that will not be covered. __init__
initializes the state of an object. It typically will set up the initial values of the class. All arguments between parenthesis when creating a new object in Python will be sent to the __init__
method (again, with a bit of nuance, as they are initially passed to __new__
). __init__
will always return None
, but typing checkers such as mypy
tend to understand this and not require a return type. The sole purpose of this dunder method is to initialize the instance.
class Car:
def __init__(self, make: str = "Honda", model: str = "Civic", year: int = 0):
self.make = make
self.model = model
self.year = year
def __repr__(self) -> str:
return f"Car(make={self.make}, model={self.model}, year={self.year})"
car1 = Car()
car2 = Car(model="Pilot", year=2025)
print(car1) # Car(make=Honda, model=Civic, year=0)
print(car2) # Car(make=Honda, model=Pilot, year=2025)
__str__
and __repr__
These two methods control how your objects look when printed. Instead of an inscrutable memory address, ( <__main__.Money object at 0x0000020610608CD0>
, for example) we can clearly show the users and developers more information
__str__
: Controls how your object looks withstr(X)
, orprint(X)
(which implicitly callsstr
. It is intended for the end user, and may not contain all the class information.__format__
. This takes a parameter string, of the format. This is useful for allf-string
representations, or for creating custom formats. I have yet to find a real use-case for this aside from formatting customdatetime
objects.__repr__
: Controls how your object looks withrepr(X)
. This is intended for developers, and should be fully unambigious.- A neat snippet that dynamically fills out the class name and all state variables is below
def __repr__(self) -> str: attrs: str = ", ".join(f"{k}={v!r}" for k, v in self.__dict__.items()) return f"{self.__class__.__name__}({attrs})"
class Point:
def __init__(self, x: int, y: int):
self.x = x
self.y = y
def __str__(self) -> str:
return f"({self.x}, {self.y})"
def __repr__(self) -> str:
return f"Point(x={self.x}, y={self.y})"
Comparison Methods
Python supports 6 comparison methods
Literal Comparison | Comparison Operator | Dunder Method Equivalent |
---|---|---|
Equal to | == |
__eq__ |
Not Equal to | != |
__ne__ |
Less Than or Equal to | <= |
__le__ |
Less Than | < |
__lt__ |
Greater Than | > |
__gt__ |
Greater Than or Equal to | >= |
__ge__ |
However, we don’t have to implement them all! If we implement __eq__
and a rich comparison operator (__lt__
, for example), we can use the @functools.total_ordering
decorator to automatically create the rest!
from functools import total_ordering
@total_ordering
class Grade:
def __init__(self, score):
self.score = score
def __eq__(self, other: object) -> object:
if not isinstance(other, Grade):
return NotImplemented
return self.score == other.score
def __lt__(self, other: object) -> object:
if not isinstance(other, Grade):
return NotImplemented
return self.score < other.score
def __str__(self):
return f"{self.score}%"
# Now all comparisons work!
grade_a = Grade(95)
grade_b = Grade(87)
print(grade_a > grade_b) # True
print(grade_a >= grade_b) # True
print(grade_a != grade_b) # True
If you still want to define the rest of the operators yourself, you can do so with simple Boolean algebra.
def __ne__(self, other: object) -> bool:
equals_result = self.__eq__(other)
if equals_result is NotImplemented:
return NotImplemented
return not equals_result
def __ge__(self, other: object) -> bool:
lt_result = self.__lt__(other)
if lt_result is NotImplemented:
return NotImplemented
return not lt_result
Arithmetic Methods
class Int:
def __init__(self, value: int):
self.value = value
def __add__(self, other: object) -> object:
if isinstance(other, Int):
return Int(self.value + other.value)
if isinstance(other, int):
return Int(self.value + other)
return NotImplemented
def __sub__(self, other: object) -> object:
if isinstance(other, Int):
return Int(self.value - other.value)
if isinstance(other, int):
return Int(self.value - other)
return NotImplemented
def __mul__(self, other: object) -> object:
if isinstance(other, Int):
return Int(self.value * other.value)
if isinstance(other, int):
return Int(self.value * other)
return NotImplemented
def __floordiv__(self, other: object) -> object:
if isinstance(other, Int):
return Int(self.value // other.value)
if isinstance(other, int):
return Int(self.value // other)
return NotImplemented
def __repr__(self) -> str:
return f"{self.value}"
# easy to continue with rsub, rmul, and rfloordiv
def __radd__(self, other: object) -> object:
# or just return self.__add__(other) instead of all this
if isinstance(other, Int):
return Int(self.value + other.value)
if isinstance(other, int):
return Int(self.value + other)
return NotImplemented
# easy to continue with iadd, imul, and ifloordiv
def __isub__(self, other: object) -> object:
if isinstance(other, Int):
self.value -= other.value
return self
if isinstance(other, int):
self.value -= other
return self
return NotImplemented
one = Int(1)
three = Int(3)
nine = Int(9)
print(one + three) # 4
print(three - one) # 2
print(one * nine) # 9
print(nine // three) # 3
print(one + 2) # 3
print(7 + three) # 10
nine -= 4
print(nine) # 5
They overload the literal operators as follows in this table:
Literal Operator | Dunder Method Equivalent |
---|---|
X+Y | __add__ , __radd__ |
X-Y | __sub__ , __rsub__ |
X*Y | __mul__ , __rmul__ |
X/Y | __truediv__ , __rtruediv__ |
X//Y | __floordiv__ , __rfloordiv__ |
X%Y | __mod__ , __rmod__ |
X**Y | __pow__ , __rpow__ |
-X | __neg__ |
~X | __invert__ |
X += Y | __iadd__ |
X-=Y | __isub__ |
X*=Y | __imul__ |
X/=Y | __itruediv__ |
X//=Y | __ifloordiv__ |
X%=Y | __imod__ |
X**=Y | __ipow__ |
It is important to note that these are parsed left to right, so Int(3) + Int(5)
is equivalent toInt(3).__add__(Int(5))
.
If you are building classes that need to be able to handle the reflections of these, you can support the __r{name}__
equivalents (__radd__
, __rfloordiv__
, etc.). If __add__
is not defined for the first object, or doesn’t know how to handle the other
type, Python will check the other
object for an equivalent reflection method. Returning NotImplemented
signals to Python that we should try the __r{name}__
methods.
You can handle in-place modification to objects with the __i{name}__
syntax.
Collection Methods
If you want your object to behave like a list or a dictionary, you should implement some of these.
from typing import Iterable # for __iter__
class Playlist:
def __init__(self, name: str, songs: list[str] = []):
self.name = name
self.songs = songs
def __iter__(self) -> Iterable: # for Y in X
return iter(self.songs) # make it iterable by referring to the inner array
def __len__(self) -> int: # len(X)
return len(self.songs)
def __getitem__(self, index: int) -> str: # X[index]
return self.songs[index]
def __setitem__(self, index: int, name: str) -> None: # X[index] = name
self.songs[index] = name
def __contains__(self, name: str) -> bool: # name in X
return name in self.songs
def __repr__(self) -> str:
return f"Playlist(songs={self.songs!r})"
def add_song(self, name: str) -> None:
self.songs.append(name)
We can now treat our Playlist
as if it was a simple array
my_playlist = Playlist("Road Trip")
my_playlist.add_song("Bohemian Rhapsody")
my_playlist.add_song("Enter Sandman")
print(my_playlist) # Playlist(songs=['Bohemian Rhapsody', 'Enter Sandman'])
print(len(my_playlist)) # 2
print("Enter Sandman" in my_playlist) # True
print(my_playlist[0]) # Bohemian Rhapsody
# We can now iterate over it using any of the various iteration methods!
for i, song in enumerate(my_playlist):
print(f"Song {i+1}: {song}")
__iter__
is used under the hood by ALL forms of iteration (for loops, enumerate, list
/dict
comprehensions, etc.)
Advanced Iteration
We can implement iteration that is more advanced than simply returning the iter
of a state value by defining the whole iterator protocol
class Countdown:
def __init__(self, start: int = 10):
self.current = start
def __iter__(self) -> object:
return self
def __next__(self) -> None:
if self.current > 0:
value = self.current
self.current -= 1
return value
else:
raise StopIteration # Tells Python we have reached the end of the iterable
rocket_launch = Countdown(10)
for number in rocket_launch:
print(number)
Here’s what is happening behind the scenes:
Countdown.__iter__(takeoff) # Countdown(2)
print(takeoff.__next__()) # 2
print(takeoff.__next__()) # 1
print(takeoff.__next__()) # raise StopIteration
Context Managers
Context managers are an essential way to ensure that error handling is done correctly. It is how something like with open("file.txt") as f:
works. We must implement the __enter__
and __exit__
dunder methods to meet the context manager protocol
class DatabaseManger:
def __init__(self, db_name: str):
self.name = db_name
self.connection = None
def __enter__(
self,
) -> object: # in practice, this should return the connection object
self.connection = f"Connected to {self.name}"
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb) -> bool:
self.connection = None
return False # propagate exceptions
db = DatabaseManger("users.db")
with db as conn:
# exit is called automatically upon leaving this block
print(conn) # Connected to users.db
print(db.connection) # None
__enter__
returns the object following the as
.
__exit__
takes the following parameters:
- The exception class
- The exception object
- A traceback object
and returns a Boolean, whether to suppress the error or not.
Callable Objects
Want your objects to behave like functions? Implement __call__
. A canonical example is the strategy pattern:
class Multiply:
def __init__(self, factor: int | float):
self.factor = factor
def __call__(self, other: int | float) -> int | float:
return self.factor * other
double = Multiply(3)
quadruple = Multiply(4)
print(double(4)) # 12
print(quadruple(4)) # 24
Notes
NotImplemented
vs NotImplementedError
Always return NotImplemented
(not raise NotImplementedError
) when your method can't handle the other operand:
def __add__(self, other):
if isinstance(other, MyClass):
return MyClass(self.value + other.value)
return NotImplemented # Not raise NotImplementedError!
This tells Python to try the other object's reverse method (__radd__
).
Infinite Recursion with __getattribute__
Be extremely careful with __getattribute__
– it's called for EVERY attribute access:
class Dangerous:
def __getattribute__(self, name):
# This causes infinite recursion!
# return self.__dict__[name]
# Use super() instead
return super().__getattribute__(name)
Final Example
import math
from functools import total_ordering
@total_ordering
class Vector2D:
def __init__(self, x: int | float, y: int | float):
self.x = x
self.y = y
def __lt__(self, other: object) -> bool:
if isinstance(other, Vector2D):
return self.magnitude < other.magnitude
return NotImplemented
def __eq__(self, other: object) -> bool:
if isinstance(other, Vector2D):
return self.x == other.x and self.y == other.y
return NotImplemented
def __abs__(self):
return self.magnitude
def __mul__(self, other: object) -> object:
if isinstance(other, (int, float)):
return Vector2D(self.x * other, self.y * other)
return NotImplemented
def __rmul__(self, other: object) -> object:
return self.__mul__(other)
def __add__(self, other: object) -> object:
if isinstance(other, Vector2D):
return Vector2D(self.x + other.x, self.y + other.y)
return NotImplemented
def __radd__(self, other: object) -> object:
return self.__add__(other)
def __sub__(self, other: object) -> object:
if isinstance(other, Vector2D):
return Vector2D(self.x - other.x, self.y - other.y)
return NotImplemented
def __rsub__(self, other: object) -> object:
return self.__sub__(other)
def __str__(self) -> str:
return f"<{self.x}, {self.y}>"
def __repr__(self) -> str:
return f"Vector2D(x={self.x}, y={self.y})"
@property
def magnitude(self):
return math.sqrt(self.x**2 + self.y**2)
v1 = Vector2D(3, 4)
v2 = Vector2D(1, 2)
print(v1 + v2) # <4, 6>
print(v1 * 2) # <6, 8>
print(2 * v1) # <6, 8>
print(abs(v1)) # 5.0
print(v1 > v2) # True
Conclusion
Magic methods or dunder methods provide a powerful way to make your Python classes behave like built-in types. Good magic methods make code more readable and intuitive. Instead of vector.add(other_vector)
, you can write vector + other_vector
. Instead of playlist.get_length()
, you can use len(playlist)
. The best magic methods happen when you don’t even notice you are using them, objects just work.