MagicDict

If you write software in Python you come to a point where you are testing a piece of code that expects a more or less elaborate dictionary as an argument to a function. As a good software developer we want that code properly tested but we want to use minimal fixtures to accomplish that.

So, I was looking for something that behaves like a dictionary, that you can give explicit return values for specific keys and that will give you some sort of a “default” return value when you try to access an “unknown” item (I don’t care what as long as there is no Exception raised (e.g.

KeyError

 )).

My first thought was “why not use MagicMock?” … it’s a useful tool in so many situations.

from mock import MagicMock
m = MagicMock(foo="bar")

But using MagicMock where dict is expected yields unexpected results.

>>> # this works as expected
>>> m.foo
'bar'
>>> # but this doesn't do what you'd expect
>>> m["foo"]
<MagicMock name='mock.__getitem__()' id='4396280016'>

First of all attribute and item access are treated differently. You setup MagicMock using key word arguments (i.e. “dict syntax”), but have to use attributes (i.e. “object syntax”) to access them.

Then I thought to yourself “why not mess with the magic methods?”

__getitem__

  and 

__getattr__

  expect the same arguments anyway. So this should work:

m = MagicMock(foo="bar")
m.__getitem__.side_effect = m.__getattr__

Well? …

>>> m.foo
'bar'
>>> m["foo"]
<MagicMock name='mock.foo' id='4554363920'>

… No!

By this time I thought “I can’t be the first to need this” and started searching in the docs and sure enough they provide an example for this case.

d = dict(foo="bar")

m = MagicMock()
m.__getitem__.side_effect = d.__getitem__

Does it work? …

>>> m["foo"]
'bar'
>>> m["bar"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../env/lib/python2.7/site-packages/mock.py", line 955, in __call__
    return _mock_self._mock_call(*args, **kwargs)
  File ".../env/lib/python2.7/site-packages/mock.py", line 1018, in _mock_call
    ret_val = effect(*args, **kwargs)
KeyError: 'bar'

Well, yes and no. It works as long as you only access those items that you have defined to be in the dictionary. If you try to access any “unknown” item you get a

KeyError

 .

After trying out different things the simplest answer to accomplish what I set out to do seems to be sub-classing defaultdict.

from collections import defaultdict

class MagicDict(defaultdict):
    def __missing__(self, key):
        result = self[key] = MagicDict()
        return result

And? …

>>> m["foo"]
'bar'
>>> m["bar"]
defaultdict(None, {})
>>> m.foo
Traceback (most recent call last):
&nbsp; File "<stdin>", line 1, in <module>
AttributeError: 'MagicDict' object has no attribute 'foo'

Indeed, it is. 😀

Well, not quite. There are still a few comfort features missing (e.g. a proper

__repr__

). The whole, improved and tested code can be found in this Gist:

MagicMock With Spec

Thanks to @immoralist I’ve learned a new Python testing trick. I didn’t know about the “spec” argument for MagicMock. m(
Let’s see an example:

from mock import MagicMock

class SomeModel(object):
    foo = "f**"
    bar = "b**"

m = MagicMock(spec=SomeModel)

Here we create a mock object which mimics the interface of 

SomeModel

  as we would expect, returning mock values for things we access.

>>> m.foo
<MagicMock name='mock.foo' id='4506756880'>
>>> m.bar
<MagicMock name='mock.bar' id='4506754192'>

Let’s see what happens if we call something else:

>>> m.baz
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../env/lib/python2.7/site-packages/mock.py", line 658, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'baz'

It will fail loudly while a mock object without a spec would have returned a mock value as it did in the previous example.

But the magic doesn’t end there. You can still set additional attributes/methods “by hand” and have them not fail even if they aren’t part of the original spec.

>>> m.baz = "bazzzzz"
>>> m.baz
'bazzzzz'

Learning new things makes me happy. 😀