Advanced Classes

This tutorial focuses on the invocation of decorators, dunder methods and demystifies the operators in Python.

Decorators

Consider the following modification of our Student class:

class Student:
    """Stores information about students."""

    def __init__(self, name):
        self.name = name
        self.homework_grades = []
        self.exam_grades = []
        self.extra_credit = None

    @classmethod
    def from_dict(cls, sdict):
        """Initialize a Student using the given dictionary."""
        student = cls(sdict['name'])
        if 'homework_grades' in sdict:
            student.homework_grades = sdict['homework_grades']
        if 'exam_grades' in sdict:
            student.exam_grades = sdict['exam_grades']
        if 'extra_credit' in sdict:
            student.extra_credit = sdict['extra_credit']
        return student

    @staticmethod
    def avg(values):
        return sum(values)/float(len(values))

    @property
    def homework_avg(self):
        """Average homework grade."""
        return self.avg(self.homework_grades)

    @property
    def exam_avg(self):
        """Average exam grade."""
        return self.avg(self.exam_grades)

    @property
    def gpa(self):
        """GPA (on a 4.0 scale)."""
        scores = [self.homework_avg, self.exam_avg]
        if self.extra_credit is not None:
            scores.append(self.extra_credit)
        return 4.0 * self.avg(scores)/100.

What are all of those things with @ before the function definitions? And why don’t the from_dict and avg methods not have self as their first arguments?!?

The names beginning with @, such as @classmethod, are called decorators. Essentially, decorators are wrappers around functions that modify the behavior of that function. They can be used on any Python function, but they are most often seen in classes.

Using and creating decorators is a large topic itself, the details of which we won’t get into here (if you’re interested, see here for an excellent tutorial). However, there are three predefined decorators in the Python builtins that often come up in class definitions that I want highlight here: @classmethod, @staticmethod, and @property.

@classmethod

The @classmethod decorator modifies methods so that instead of taking an instance of a class as the first argument (what we normally call self) it takes the class itself (which we normally call cls). This is typically used to provide a way to instantiate a class using alternate input arguments than what is defined in __init__. For example, say we had a dictionary that specifies all of the information about Susie:

susie_dict = {}
susie_dict['name'] = 'Susie'
susie_dict['homework_grades'] = [80., 73., 77., 50., 0.]
susie_dict['exam_grades'] = [70., 63., 50.]

We can now instantiate a Student representation of Susie using the from_dict method:

susie = Student.from_dict(susie_dict)

print(susie.name)
print(susie.homework_grades)
print(susie.exam_grades)

Susie
[80.0, 73.0, 77.0, 50.0, 0.0]
[70.0, 63.0, 50.0]

Notice that when we used from_dict we only provided the dictionary, even though the definition of from_dict had two arguments, cls and sdict. As with normal methods, @classmethod automatically adds in the class as the first argument.

@staticmethod

The @staticmethod decorator modifies methods so that the class instance self is not automatically added when the method is called. As a result, we do not need to reserve the first argument in the definition of a @staticmethod. We see that in the above example: avg takes a single argument values, which is a list of values to calculate an average for.

Since methods wrapped with @staticmethod do not do any automatic substitutions, it’s possible to use them without needing to create a class instance. For example:

Student.avg([60., 70.])

65.0

In fact, we could have just defined avg outside of Student in the global namespace. In that case we would of called avg(...) instead of self.avg(...) inside of Student.

So why use @staticmethod? Its main purpose is to provide a function that is heavily used by a class, but may have little meaning outside of the class. This can make code a easier to read and understand (”syntactic sugar”). Basically, @staticmethod is a way of logically organizing functions.

@property

The @property decorator is another form of syntactic sugar that modifies how a method is called. Basically, it makes a method look like an attribute.

For example, in the above homework_avg, exam_avg, and gpa were all methods. Normally you would call them like susie.gpa() (as we did in the classes section). However, because we stuck the @property decorator on each of these methods, we instead do:

susie.gpa

2.34

In other words, we no longer include the () after the name. Note that @property only works with methods that take no arguments.

The @property is used when we want to run some additional code under the hood when an attribute is accessed. It is typically paired with a “setter”, which allows us to also run some code when an attribute is set. For example:

class Star:
    _mass = None

    @property
    def mass(self):
        if self._mass is None:
            raise ValueError("no mass set")
        return self._mass

    @mass.setter
    def mass(self, value):
        if value <= 0:
            raise ValueError("mass must be > 0")
        self._mass = value

star = Star()

print(star.mass)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[8], line 1
----> 1 print(star.mass)

Cell In[6], line 7, in Star.mass(self)
      4     @property
      5     def mass(self):
      6         if self._mass is None:
----> 7             raise ValueError("no mass set")
      8         return self._mass

ValueError: no mass set

star.mass = 10.
print(star.mass)

10.0

star2 = Star()
star2.mass = -3

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[10], line 2
      1 star2 = Star()
----> 2 star2.mass = -3

Cell In[6], line 13, in Star.mass(self, value)
     10     @mass.setter
     11     def mass(self, value):
     12         if value <= 0:
---> 13             raise ValueError("mass must be > 0")
     14         self._mass = value

ValueError: mass must be > 0

Dunder

Dunder (Double underscore) methods __meth__ are names which are reserved for Python only and should not be invented by us (implemented; yes). Just to be precise, __meth is fine.

These methods are used to delegate the actual operator call. When we invoke any operator (+, ==, but also () and so on), the object that this is applied on is checked for a corresponding method. For the +, the object (on the left) is checked for a __add__ method. If this is not found or return a NotImplemented either alternatives are tried or an error is raised if all possibilities are tried.

Alternatives involve in this case to call the __radd__ (right add) on the object on the right IF the objects are of different types.

len

The len method simply checks if there is a __len__ implemented.

str

To have a nice, representable, human readable string, __str__ should be implemented. There is a similar one, which is __repr__. This is also a string representation of the object, yet more targeted towards the developers.

If no __str__ is provided, it falls back to __repr__, which, if not provided, uses a default implementation.

class Name:
    def __init__(self, name):
        self.name = name


class NameRepr(Name):
    def __repr__(self):
        return self.name


class NameStr(Name):
    def __str__(self):
        return f'I am {self.name}'


class NameStrRepr(NameStr, NameRepr):
    pass

Exercise: try it out by using str(...) and repr(...)

Callable

In Python, a callable is any object that can be called. Calling an object means to have (...) attached behind it. This operator looks for a __call__ method.

class Callable:
    def __call__(self, *args, **kwargs):
        print(f"called with args {args} and kwargs {kwargs}")


class NotCallable:
    pass

call = Callable()
noncall = NotCallable()

call()

called with args () and kwargs {}

try:
    noncall()
except TypeError as error:
    print(error)

'NotCallable' object is not callable

TypeError: 'NotCallable' object is not callable translates to has no __call__ method

Indexing (iterating)

There are a few methods when it comes down to iteration. However, we won’t go into these details but rather look at the normal indexing. That is controlled via __getitem__ and __setitem__ and invoked with the [] operator.

class Storage:
    def __init__(self, name):
        self.name = name
        self.container = [1, 5, 4]  # just for demonstration

    def __getitem__(self, index):
        print(f"getitem of {self.name} invoked with index {index}")
        return self.container[index]

    def __setitem__(self, index, item):
        print(f"setitem of {self.name} invoked with index {index} and item {item}")
        self.container[index] = item

storage = Storage('one')

storage[2]

getitem of one invoked with index 2

4

storage[2] = 3

setitem of one invoked with index 2 and item 3

self

What is actually self? Nothing else than the object itself. However, we can rename it however we like.

Read the following well If an instance is create of a class and a method is called on that instance, the first argument to the method is the instance itself. Fullstop

What are the consequences of this?

class A:
    def __init__(self, value):
        self.value = value

    def add(self, y):
        return self.value + y.value

a = A(4)
b = A(38)

a.add(b)

42

A.add(a, b)

42

The latter works as well! Why not? add is a method that we call and we give it two arguments. Forgetting about class dynamics, it makes actually complete sense.

Naming conventions

In Python, there are a number of conventions for naming variables, functions, and classes. These are not enforced by the Python interpreter, but are used to make code more readable and understandable.

1. Class names should be in CamelCase (e.g., MyClass).

2. Constant names should be in all caps with underscores separating words (e.g., MY_CONSTANT).

3. Everything else should be in snake_case, that is, all lowercase with underscores separating words (e.g., my_function).

For more details, see PEP 8, the official Python style guide.

Methods of classes have their own naming conventions, some of which have an actual effect on how the method is treated by Python. There are four types:

• Public methods are those that are part of the class’s public API. These should be in snake_case, for example, A.my_method().

• Private methods are those that are only to be used within the class. These should begin with an underscore, for example, _my_private_method(). This is a convention only; Python does not enforce privacy. These methods should not be used as they can be changed without notice. The use-case to actually use them is usually to “hack” something.

• Double private are methods that begin and end with double underscores. These are not only private but their name is mangled (with a _class prepended, where class is the actual class name), which is a way of making it difficult to override a method in a subclass. They can, technically, still be called, but it is made purposefully difficult. For example, __my_double_private_method() (calling it from the outside will be _MyClass__my_double_private_method()).

• Dunder or magic methods are methods that are called by Python in response to certain events. They always begin and end with double underscores. For example, __init__ is called when an instance of a class is created. Others include __str__, __add__, __len__, etc. For more on these, see here. They are reserved for Python’s internal use and should not be defined by the user.

class A:
    def plus(self):
        print("plus")

    def _plus(self):
        print("_plus")

    def __plus(self):
        print("__plus")

    def __init__(self):
        print("init")

    def __str__(self):
        return "HELLO"

a = A()
a.plus()
a._plus()
a._A__plus()  # notice the mangled name! a.__plus() would not work

init
plus
_plus
__plus

class NamedValue:
    def __init__(self, name):
        self.name = name


class ValueLeft(NamedValue):
    def __add__(self, other):
        print(f"add called on {self.name}")
        return 42


class ValueRight(NamedValue):
    def __radd__(self, other):
        print(f"radd called on {self.name}")
        return 24


class Value(ValueRight, ValueLeft):
    pass

Exercise: which one can we add and which raise an error? Think or try it out!

valleft = ValueLeft('val left')
valleft2 = ValueLeft('val left2')

valleft + valleft2

add called on val left

42

Danger zone

The following is only for fun and should not be used in real live, except you do really know what you’re doing and at least two independent colleagues agree that this is the right way to go

We have seen that basically everything is an operator and it has a dunder method. Everything? Quiz: what did we miss?

Solution: the . the access operator. Yes, you guessed right. Let’s override it

First, where are actually all the attributes stored in a class? Answer: in the __dict__ attribute.

a.__dict__

{}

Next quiz: where are the methods (remark open) stored?

a.__class__.__dict__

mappingproxy({'__module__': '__main__',
              'plus': <function __main__.A.plus(self)>,
              '_plus': <function __main__.A._plus(self)>,
              '_A__plus': <function __main__.A.__plus(self)>,
              '__init__': <function __main__.A.__init__(self)>,
              '__str__': <function __main__.A.__str__(self)>,
              '__dict__': <attribute '__dict__' of 'A' objects>,
              '__weakref__': <attribute '__weakref__' of 'A' objects>,
              '__doc__': None})

A.__dict__

mappingproxy({'__module__': '__main__',
              'plus': <function __main__.A.plus(self)>,
              '_plus': <function __main__.A._plus(self)>,
              '_A__plus': <function __main__.A.__plus(self)>,
              '__init__': <function __main__.A.__init__(self)>,
              '__str__': <function __main__.A.__str__(self)>,
              '__dict__': <attribute '__dict__' of 'A' objects>,
              '__weakref__': <attribute '__weakref__' of 'A' objects>,
              '__doc__': None})

To be clear, there is nothing special about a value attribute and a method: the value attribute happened to be set on the instance while the method happened to be set on the class. But we can have class attributes as well as (not really occuring in reality though) instance methods.

Disclaimer: the following is EXTREMELY BAD CODING practices and should NEVER be seen in ANY real used code

class GetAndSet:
    def __init__(self):
        self.values = [1, 2, 3, 4, 5]

    def add(self, y):
        return self.values[0] + y.values[1]

    def __getattr__(self, name):
        if name in ('add', 'addition'):
            return self.add
        if name == 'hello':
            print('I am 42')

    # we omit the __setattr__, but the game is the same

get = GetAndSet()

get.add(get)

3

get.addition(get)

3

get.hello

I am 42

We can also provoke the same behavior by using the function getattr (or setattr respectively)

getattr(get, 'hello')

I am 42

get.hi

Quiz: why the above?

Answer: Because the __getattr__ that we called returns None (as any function/method does without an explicit return)

import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

And then there is the maybe most important sentence of all in Python:

We’re all adults here.

Behave like one when coding ;)