DEV Community

Derek Enos
Derek Enos

Posted on

Python Charming with Toolz

Toolz is an extremely useful collection of utility functions that can help you do things that you do all the time, but more explicitly, and often with less code.

I'm going to highlight a few of the functions that I use the most, but there are many others, and they're all solid gold.

Create a new dict from an existing dict with some items excluded

Given an original dict:

original = {
    'A': 1,
    'B': 2,
    'C': 3,
}
Enter fullscreen mode Exit fullscreen mode

Filtering items by key

To create a new dict from original while excluding the item with key='C', without toolz, I'd probably do:

new = {
    k: v
    for k, v in original.items()
    if k != 'C'
}
Enter fullscreen mode Exit fullscreen mode

Using dicttoolz.keyfilter, this becomes:

new = keyfilter(lambda k: k != 'C', original)
Enter fullscreen mode Exit fullscreen mode

What I really love about this keyfilter solution, aside from being more concise, functional, etc., is that its name clearly tells you what's going on.

Filtering items by value

Given a helper function called is_even that returns a bool indicating whether the single numeric argument is even:

is_even = lambda x: x % 2 == 0
Enter fullscreen mode Exit fullscreen mode

Or, as not a lambda:

def is_even(x):
    return x % 2 == 0
Enter fullscreen mode Exit fullscreen mode

To create a new dict from original while including only the even values, without toolz, I'd probably do:

new = {
    k: v
    for k, v in original.items()
    if is_even(v)
}
Enter fullscreen mode Exit fullscreen mode

Using dicttoolz.valfilter, this becomes:

new = valfilter(is_even, original)
Enter fullscreen mode Exit fullscreen mode

For a single key, get the value from each dict in a list of dicts

Given a list of dicts representing users:

users = [
    { 'name': 'User A',
      'age': 30,
    }, 
    { 'name': 'User B',
      'age': 32,
    }, 
    { 'name': 'User C',
      'age': 41,
    }, 
    { 'name': 'User D',
      'age': 43,
    }, 
]

Enter fullscreen mode Exit fullscreen mode

To get just the ages, without toolz, I'd do:

ages = [ user['age'] for user in users ]
Enter fullscreen mode Exit fullscreen mode

Using itertoolz.pluck, this becomes:

ages = pluck('age', users)
Enter fullscreen mode Exit fullscreen mode

Note that this ^ (like some other toolz functions) actually returns a generator-type itertools.imap object, which is fine if you're going to iterate over it, but if you want to index into it like ages[0], you'll need to convert it to a list (or tuple):

>>> ages[0]
TypeError: 'itertools.imap' object has no attribute '__getitem__'

>>> ages = list(pluck('age', users))
>>> ages[0]
30
Enter fullscreen mode Exit fullscreen mode

You can also pluck multiple keys at once by specifying them as a list:

age_name_pairs = pluck(['age', 'name'], users)
Enter fullscreen mode Exit fullscreen mode

Which looks like:

>>> list(age_name_pairs)
[(30, 'User A'), (32, 'User B'), (41, 'User C'), (43, 'User D')]
Enter fullscreen mode Exit fullscreen mode

Group a list of dicts by some criteria

Given the same list of user dicts as before:

users = [
    { 'name': 'User A',
      'age': 30,
    }, 
    { 'name': 'User B',
      'age': 32,
    }, 
    { 'name': 'User C',
      'age': 41,
    }, 
    { 'name': 'User D',
      'age': 43,
    }, 
]

Enter fullscreen mode Exit fullscreen mode

If I wanted to group the users into decade-resolution age groups (i.e. 30s, 40s, etc.), without toolz maybe I'd do something like:

age_decade_users_map = defaultdict(list)
for user in users:
    age_decade = int(user['age'] / 10) * 10
    age_decade_users_map[age_decade].append(user)

Enter fullscreen mode Exit fullscreen mode

This produces:

>>> dict(age_decade_users_map)
{30: [{'age': 30, 'name': 'User A'}, {'age': 32, 'name': 'User B'}],
 40: [{'age': 41, 'name': 'User C'}, {'age': 43, 'name': 'User D'}]}
Enter fullscreen mode Exit fullscreen mode

Using itertoolz.groupby, this can be accomplished with:

age_decade_users_map = groupby(
    lambda user: int(user['age'] / 10) * 10,
    users
)
Enter fullscreen mode Exit fullscreen mode

Honorable mention goes to:

  • itertoolz.first - get the first element from an iterable (even sets!)

  • itertoolz.partitional_all - split an iterable into tuples of a specified max length

  • dicttoolz.assoc - get a copy of a dict with a specified item added

  • dicttoolz.dissoc - get a copy of a dict with specified keys removed (similar in function to my keyfilter example but a better way to do it)

  • dicttoolz.get_in - like dict.get() but with support for multi-depth keys, e.g. get_in(['a', 'b'], {'a': {'b': 1}}) = 1

Happy toolzing!

Top comments (0)