DEV Community

Cover image for 🐍 Beware of Python dict.get()
Pierre
Pierre

Posted on • Updated on • Originally published at medium.freecodecamp.org

🐍 Beware of Python dict.get()

If you think that value = my_dict.get('my_key', 'default_value') is equivalent
to value = my_dict.get('my_key') or 'default_value' you should probably read this πŸ˜ƒ. If you know why it’s not the same, then you probably won’t learn something here.

The good thing:

As anyone using Python 3 should know, the dict API is very clear and simple. I can declare a dict like this:

my_car = {'wheels': 4, 'brand': 'Tesla'}
Enter fullscreen mode Exit fullscreen mode

It is simple, quick and easy. Retrieving values is as easy:

my_car.get('brand')
>'Tesla'
my_car['brand']
>'Tesla'
Enter fullscreen mode Exit fullscreen mode

But to retrieve values I prefer the .get() for two reasons. First, there will be no exceptions raised if the key you want to access is not here (it will return None). Second, you can pass a default value to the method that will be returned if the key is not present in the dict :

my_car['color']
>KeyError: 'color'

my_car.get('color')
>

my_car.get('color', 'black')
>'black'
Enter fullscreen mode Exit fullscreen mode

And the tricky one:

Now I’m going to show you what happened in the real world while fixing a bug for ShopToList in a method I wrote that uses a lib that extracts metadata from an HTML page (in this case an e-commerce page).

To make things short, the data I expected should look like this (simplified example):

data_from_extruct = {    
    'title': 't-shirt',    
    'brand': 'french-rocket',    
    'color': 'green',    
    'offer': {        
          'amount': 20,        
          'currency': '€'
    }
}
Enter fullscreen mode Exit fullscreen mode

The easiest way to get the price from this data is:

    price_from_extruct = data_from_extruct['offer']['amount']
    > 20
Enter fullscreen mode Exit fullscreen mode

But as I said before, this solution is not robust at all. This is the real world, and in the real world the data from extruct will not always come with an offer and with a price in that offer. A better way to do this is to use dict.get:

price_from_extruct = data_from_extruct.get('offer').get('amount')
Enter fullscreen mode Exit fullscreen mode

This is still not good enough because if there is no offer in the data, you will try to perform the second .get('amount') on None and it will raise an error. A way to avoid that is to do:

price_from_extruct = data_from_extruct.get('offer',{}).get('amount')
Enter fullscreen mode Exit fullscreen mode

Here, if we don’t have offer in the data, the first get will return {} (empty dict) instead of None, and then the second get will be performing against an empty dict and will return None . All is great, it seems that we have a robust way to extract the price fromthe data that is not consistently formatted. Of course sometimes the value will be none but at least this code should never break.

Well, we are wrong. The catch comes from the behavior of the default parameter. Remember that the default value will be returned if, and only if, the key is absent from the dict.

What it means is that if the data you receive looks like this:

data_from_extruct = {    
    'title': 't-shirt',    
    'brand': 'french-rocket',    
    'color': 'green',    
    'offer': None
}
Enter fullscreen mode Exit fullscreen mode

Then the previous snippet will break:

price_from_extruct = data_from_extruct.get('offer',{}).get('amount')
> AttributeError: 'NoneType' object has no attribute 'get'
Enter fullscreen mode Exit fullscreen mode

Here the default value of get('offer', {}) was not returned because the key offer was in the dict. It was just set to None.

Of course Python is awesome so there are lots of simple way to fix this. The following snippet is just one of them:

offers_from_extruct = data_from_extruct.get('offer') or {}
price_from_extruct = offers_from_extruct.get('amount')
Enter fullscreen mode Exit fullscreen mode

Of course, this can also break if the content of offer is a list, for example. But for the sake of the example, we will stop here.

Thank you for reading

I hope this short post will help you save time in the future. I wish I knew this before spending a shameful amount of time trying to fix a certain bug this week.

Please tell me in the comments if you liked the post and don't forget to subscribe to my newsletter, there is more to come (And you'll also get the first chapters of my next ebook for free 😎).

If you like JS, I've just published something you might like.

And if you prefer git, I got you covered.

Top comments (6)

Collapse
 
tobiassn profile image
Tobias SN

Wouldn’t or also fail if the value was something like 0 or False?

Collapse
 
dan_m_sullivan profile image
Daniel M. Sullivan
In [1]: 0 or {}
Out[1]: {}

In [2]: False or {}
Out[2]: {}
Collapse
 
therealdarkmage profile image
darkmage

This adds a strong technique to my repertoire and also some "gotchas" to look out for. Thank you!

Collapse
 
daolf profile image
Pierre

You are welcome! Glad you liked it.

Collapse
 
qm3ster profile image
Mihail Malo

That's pretty horrifying.
What's the robust solution here? Using a validation lib that will verify the whole data against a schema, and then have type annotations for the validated object?

Collapse
 
dwd profile image
Dave Cridland

Checking the return value properly. Or, just going for it and catching exceptions:

try:
    price_from_extruct = data_from_extruct['offer']['amount']
except:
    price_from_extruct = 0  # If you really wanted to do this.

The thing is, I find exceptions a good way of handling complex and arbitrary error flows. If you don't have a good story for what to do, trying to avoid the exceptions is just causing more headaches later.