Mastering Higher-Order Functions

Introduction

A higher-order function in computer science is one that accomplishes at least one of the following:

• Accepts one or more functions as arguments.
• Returns a function as a result

These functions are quite beneficial for writing more abstract, reusable code.

In this post, we will learn about some of the most popular and often used higher-order functions in Python and how to write our own.

One of the most popular applications of higher-order functions in Python is data creation and manipulation. Higher-order functions such as map(), filter(), and reduce() are examples of built-in functions that may be used to transform and process data in several ways.

Let us go over them one by one.

Map

The map function accepts a function and an iterable as inputs and applies the function to each element of the iterable before producing a new iterable with the results. This is handy for performing a specific action on many data points at once without the need for a loop.

Take the problem of calculating the square of each number in the provided list, for instance. The simplest way to achieve this using a loop will be as shown below:

numbers = [1, 2, 3, 4, 5]
squares = []
for i in numbers:
    squares.append(i**2)
print(squares)  
# Output: [1, 4, 9, 16, 25]

Let's see how we can accomplish the same thing with map() instead:

numbers = [1, 2, 3, 4, 5]
squares = map(lambda x: x**2, numbers)
print(list(squares)) 
# Output: [1, 4, 9, 16, 25]

To clarify, the map takes iterable numbers and applies function lambda x: x**2 to it to produce a new iterable squares. Since it is iterable, we typecast the squares to a list before printing the result.

Filter

The filter function, on the other hand, takes a function and an iterable as inputs and produces a new iterable containing only the items of the original iterable for which the function returns True. This is useful for filtering big datasets and obtaining only the important information.

Consider the task where we want to return only even integers from the list. Using a loop, we can do this by doing the following:

numbers = [1,2,3,4,5,6,7,8,9,10]
even_numbers = []
for i in numbers:
    if i % 2 == 0:
        even_numbers.append(i)
print(even_numbers)
# Output: [2, 4, 6, 8, 10]

Now let us do the same using filter():

numbers = [1,2,3,4,5,6,7,8,9,10]
even_numbers = filter(lambda x: x % 2 == 0, numbers)
print(list(even_numbers))
# Output: [2, 4, 6, 8, 10]

We achieve the same result in one-liner clear code.

Reduce

The reduce function is a little different; for a starter, it is not a built-in function like the map and filter. We need to import it from the functools module. The reduce accepts a function and an iterable as parameters and, to reduce the iterable to a single value, cumulatively applies the function to each member of the iterable from left to right. This is efficient for performing complex calculations on huge datasets.

Assume you have a collection of family bills and your assignment is to determine the overall spending. The basic program will look like this:

bills = [100,200,300,400,500]
overall_spent = 0
for i in bills:
    overall_spent += i
print(overall_spent)
# Ouptut: 1500

This is how we can do it using reduce(),

from functools import reduce
bills = [100,200,300,400,500]
overall_spent = product = reduce(lambda x, y: x+y, bills)
print(overall_spent)
# Output: 1500

Custom higher-order function

Python also allows programmers to write their higher-order functions. This may be achieved by writing a function that accepts one or more other functions as arguments or by returning a function as a result.

These custom higher-order functions may be utilized to build more complex and efficient applications without the need for repeated or redundant coding.

Check out the following code snippet:

def make_greeting(greeting):
    def greet(name):
        return f"{greeting}, {name}!"
    return greet

hello = make_greeting("Hello")
print(hello("Jon")) # Output: "Hello, Jon!"

goodbye = make_greeting("Goodbye")
print(goodbye("Jane")) # Output: "Goodbye, Jane!"

Here, we wrote a custom higher-order function called make greeting that accepts a greeting as an argument and produces a new function called greet that accepts a name as an argument and returns a greeting message. We then built two welcome function instances, one for "Hello" and one for "Goodbye."

Conclusion

Higher-order functions are commonly used in software development to produce reusable, modular code. They are incredibly helpful and widely used in data science and machine learning. Consider working with enormous datasets and wanting to do preprocessing to clean data; here is where functions like map, reduce, and the filter comes into play.

Therefore, it is crucial to understand its fundamentals if one hopes to flourish in the fields of software engineering or data science. I hope this post is informative and gives you a good insight into higher-order functions. Happy Learning!