ChatGPT-4 outperformed 90% of developers

The title may sound bold and clickbait-ish, but it is what it is...

A few days ago I stumbled upon a post on LinkedIn (unfortunately, I didn't save the link) in which a programmer demonstrated his solution to a LeetCode challenge. The code was fine and the post got dozens of likes, which it undoubtedly deserved. I decided to feed the requirements to ChatGPT-4 and see if it could understand and solve the problem. The outcome was... interesting and I spent some time experimenting with the AI, modifying the prompt to see how it affected the results.

The Challenge

First, I would like you to take a look at the LeetCode problem and attempt to solve it. Take your time, give it some thought. Then, google it. Yes, search the internet for the best solution.

Given two integer arrays pushed and popped each with distinct values, return true if this could have been the result of a sequence of push and pop operations on an initially empty stack, or false otherwise.

Example 1:

Input: pushed = [1,2,3,4,5], popped = [4,5,3,2,1]
Output: true
Explanation: We might do the following sequence:
push(1), push(2), push(3), push(4),
pop() -> 4,
push(5),
pop() -> 5, pop() -> 3, pop() -> 2, pop() -> 1

Example 2:

Input: pushed = [1,2,3,4,5], popped = [4,3,5,1,2]
Output: false
Explanation: 1 cannot be popped before 2.

Constraints:

• 1 <= pushed.length <= 1000

• 0 <= pushed[i] <= 1000

• All the elements of pushed are unique.

• popped.length == pushed.length

• popped is a permutation of pushed.

Done?

Alright, remember I asked you to brainstorm and google it? So if the machine-generated solution happens to be more efficient, the argument "it's not fair because it was trained on data from the Internet" is not accepted 😉

Release the Kraken

When I initially asked ChatGPT to tackle the challenge, I copy-pasted the task from LeetCode. However, in this case, it might indeed seem like the AI pulled the solution from the depths of its "memory" without much "thinking". So let's rephrase the requirements and see if the machine can handle it. Here's the prompt I used:

The output:

Not bad, and according to LeetCode statistics, this is an average solution, more or less. Now let's push our beloved AI a bit further and ask it to enhance the code:

This is where things become really interesting! Not only did it offer an elegant solution without allocating a stack, but it also explained a potential issue that could arise from this optimization. Let's check it out on LeetCode:

The numbers speak for themselves, right? Of course, some might argue that LeetCode's benchmark is inconsistent, which is a valid point. Nevertheless, the fact remains that the AI instantly produced a decent solution and improved it in a way that not every software engineer can do.

What's Next?

So, are our jobs safe?

Yes 😅 (unless you work for Elon Musk)

To draw an analogy, today's ChatGPT (and other AI assistants like GitHub Copilot) is like an electric tool that can drill holes, drive screws, saw and more. However, if you just place it on the ground, it won't build a house for you. What is more, unskilled individuals might even harm themselves with this tool. On the other hand, experienced engineers can use the power and potential of AI to significantly improve the quality of their work.

That's it for now. Cheers!