What is Algorithmic Efficiency?: An Introduction to Asymptotic Analysis.

Algorithms are at the heart of computer science and programming. They are a set of instructions that a computer follows to perform a specific task.

It's like a recipe that helps you bake a cake or cook a meal, but for computers. Just like there are different recipes to cook a meal, there are also different ways to solve a problem in programming and it is in the interest of the programmer to pick the most efficient approach.

Algorithmic efficiency is the measurement of the performance of an algorithm in terms of time and space/memory complexity. Simply put, an efficient/ideal algorithm takes up the least possible memory and executes it in the least possible time.

When analyzing algorithms, it's not enough to simply measure how long an algorithm takes to run on a specific input. Instead, we need to have an understanding of how an algorithm's performance scales as the input size grows.

This is where Asymptotic analysis comes in. It allows us to analyze the performance of an algorithm as the input size grows larger and larger. By using asymptotic analysis, we can compare different algorithms and make informed decisions about which algorithm to use in a given situation. We can also identify bottlenecks in an algorithm's performance and find ways to optimize it.

What is Asymptotic Analysis?

Asymptotic analysis is a technique that is used to analyze the behavior and performance of an algorithm as the input size grows larger and larger.

To understand this, let's consider a simple example:

Imagine you were given an array of numbers and told to find a number within that list and deduce its position.

Linear Search Algorithm

One way you can solve this task is to go into the array and check the position of every value in the list until we get to the value we are looking for. And if we don't find the value only then can we confirm that the element does not exist in the array.

This is called the linear search algorithm and here is an example of its implementation in python:

def linear_search(array, target):
    for i in range(len(array)):
        if array[i] == target:
            return i
    return None

In the above code, the function linear_search function takes an array and a target element as inputs. It then iterates through each element in the array until it finds the target element, and returns the index position of the target element. If the target element is not found in the array, the function returns None.

Given an array of 5 elements, this might look like a very good algorithm as it won't take much time to find the target element. But if we keep increasing the size of the array, we would find out that there would be a proportionate increase in the amount of time it will take for our algorithm to execute the task.

A graphical interpretation of this algorithm will look like this:

As we can see, this is not an efficient algorithm because as the input size increases, it takes more and more time to execute.

Here's a better way to search -

Binary Search Algorithm

This algorithm begins by comparing the target value to the middle element of the array. If the middle element is the target value, the search is complete. Otherwise, the algorithm determines whether the target value is greater or less than the middle element, and discards half of the list based on this comparison. The process is repeated with the remaining half of the list until the target value is found or the list is empty

For example, let's say we have a sorted list of numbers [0, 1, 3, 4, 5, 7, 9, 11, 13] and we want to search for the value 7.

1. The algorithm starts by comparing 7 to the middle element, which is 5.

2. Since 7 is greater than 5, the algorithm discards the first half of the list and repeats the process with the remaining half [7, 9, 11, 13].

3. The algorithm now compares 7 to the middle element of this new list, which is 9.

4. Since 7 is less than 9, the algorithm discards the second half of the list and repeats the process with the remaining half [7].

5. The algorithm has now found the target value and the search is complete.

Note: In a binary search, the elements must be arranged in sorted order.

Here's an implementation of the binary search algorithm in python:

def binary_search(list, item):
    low = 0
    high = len(list) - 1

    while low <= high:
        mid = int((low + high)/2)
        guess = list[mid]

        if guess == item:
            return mid
        if guess > item:
            high = mid - 1
        else:
            low = mid + 1
    return None

sorted_list = [0, 1, 3,4, 5, 7, 9, 11, 13] 

print(binary_search(sorted_list, 7))

Now let's see the graphical representation of how this algorithm compares to the linear search algorithm

As we can see, the binary search outperforms the linear search algorithm as the input size keeps increasing. This makes it a more efficient algorithm for this case.

I want you to observe something peculiar in the graph above. We can see that for small input sizes, the linear search algorithm and the binary search look to perform at the same rate. If we were to take our decision from this point, it would be a wrong and ill-informed decision. This is why we use asymptotic analysis!

Benefits of Asymptotic Analysis

As we have seen earlier, asymptotic analysis is a very important tool for analyzing the efficiency of algorithms. I've listed some of its benefits below:

1. Simplification: Asymptotic analysis allows us to simplify the performance analysis of algorithms by focusing on their growth rate as the input size increases.

2. Efficiency Comparison: Asymptotic analysis allows us to compare the efficiency of algorithms for large input sizes, regardless of the specific hardware or software environment in which they are executed.

3. Design Improvement: Asymptotic analysis can help us identify inefficiencies in algorithms and guide us in making improvements to their design, leading to faster and more scalable solutions.

4. Prediction: Asymptotic analysis can help us predict the behavior of algorithms for large input sizes, allowing us to plan for the resources needed to execute them effectively.

5. Language and platform independence: Asymptotic analysis provides a language and platform-independent way of analyzing algorithms, making it easier to communicate about their performance characteristics and compare them across different programming languages and platforms.

And that's a wrap 😃

As we have seen, Algorithmic efficiency refers to how well an algorithm performs in terms of the resources it requires, such as time and memory. Asymptotic analysis is a technique used to analyze the efficiency of an algorithm as the input size grows.

An efficient algorithm should be able to handle large inputs in a reasonable amount of time and with minimal memory usage.

If you found this article insightful, kindly like, comment, and share it. I'd like this article to reach as many people as possible.

Thank you and happy coding