Visual Studio 2022 vs Rider vs ChatGPT-4

Today I was going to post a tweet like "Hey fellow developers, use Rider/Resharper instead of good old Visual Studio because the former will find and fix some bugs for you", but then I decided to check what the latest Visual Studio 2022 can do against Rider, and while at it, I remembered our new friend ChatGPT... So instead of a short tweet, I'd like to offer you a small competition between Visual Studio 2022, Rider 2023.1 and the magical ChatGPT-4.

What we're going to do today is review a C# program and try to spot any bugs. Then we'll feed the code to our favorite IDEs (Rider and Visual Studio) to see if they can highlight any code issues. Finally, we're going to ask ChatGPT to review the code and fix all discovered bugs if any.

Shall we?

Requirements

Let's say we want to write a C# program that:

1. Generates 10 users with randomized ratings.

2. Finds users whose ratings are between X and Y.

3. If any users are found, then print:

Code

Instead of writing this program ourselves, we asked someone else to do it for us (someone less smart than modern AI systems), and here is what we got:

var allUsers = GetAllUsers();
var foundUsers = FindUsersByRating(allUsers, 0, 60);

if (foundUsers.Any())
{
    var userWithMinRating = foundUsers.OrderBy(x => x.Rating).FirstOrDefault();
    PrintUsers(foundUsers);
    Console.WriteLine($"{foundUsers.Count()} user(s) found.");
    Console.WriteLine($"Min rating: {userWithMinRating.Rating}.");
    var title = GetTitleByRating(userWithMinRating.Rating);
    Console.WriteLine($"Title: {title.ToUpper()}");
}
else
{
    Console.WriteLine("No users found.");
}

IEnumerable<User> FindUsersByRating(IEnumerable<User> users, int minRating, int maxRating)
{
    return users.Where(x => x.Rating >= minRating && x.Rating <= maxRating);
}

IEnumerable<User> GetAllUsers()
{
    var randomizer = new Random();
    return Enumerable.Range(1, 10)
                     .Select(x => new User(x, randomizer.Next(0, 100)));
}

string GetTitleByRating(int rating)
{
    return rating switch
           {
               > 0 and < 30 => "Beginner",
               >= 30 and < 80 => "Advanced",
               >= 80 => "Expert",
               _ => null
           };
}

void PrintUsers(IEnumerable<User> users)
{
    foreach (var user in users)
    {
        Console.WriteLine(user.ToString());
    }
}

record User(int Id, int Rating);

Now, let's take a moment and think about what issues this unreliable, to say the least, code has.

Finished? Most certainly, you managed to find some, but let's see if our beloved dev tools can simplify it for us and highlight issues that occasionally could escape our tired, red eyes.

Visual Studio 2022 Professional

The latest, at the time of writing, .NET IDE from Microsoft.

(The rest of the code didn't fit on the screenshot, but it doesn't matter).

As we can see, Visual Studio doesn't want to tell us anything (because it doesn't see any issues). Alright, let's move on to the next one Rider.

Rider (v 2023.1)

(Similar results to Visual Studio + Resharper)

Okay, we've got something here!

Possible multiple enumeration

The first thing Rider draws our attention to is the "Possible multiple enumeration" warning, which in this particular case is not just "possible", but an actual issue:

The GetAllUsers method returns a "lazy" IEnumerable<User> instead of actual objects, and, as a result, on lines 4, 6, 7 and 8, where we enumerate this IEnumerable<User>, new User objects will be created. Every time. In other words, when we run our program and the execution point reaches the code on line 1, no actual users will be generated. The same is true for line 2. And only when we reach line 4 (foundUsers.Any()) the actual User instances will be created, filtered and the if statement will check if we have any items left.

So far, so good, but the problem is that on line 6 the FirstOrDefault() method will initiate the process of creating and filtering new items over again. Completely new items. And the same thing happens inside of the PrintUsers method when we iterate over the IEnumerable with the foreach loop, and the last enumeration happens on line 8 when we call the Count() method.

Thus, we can see the following output:

User { Id = 6, Rating = 46 }
User { Id = 8, Rating = 70 }
0 Users found.
Min rating: 51.
Title: EXPERT.

What nonsense! We can see 2 users, but it says that none were found, then it prints a minimum rating that doesn't belong to either of the two users we saw just two lines above. To top it off, we get the EXPERT title... As we discussed, all this happens because new sets of users are being generated on lines 4, 6, 7 and 8.

Possible 'System.NullReferenceException'

Another fair point made by Rider is that userWithMinRating can be null, which can lead to a NullReferenceException being triggered on line 9:

Although it may seem like a simple and easy-to-spot bug, it's one of the most common exceptions that occur in production code.

ChatGPT

Alright, now it's time to test-drive the extremely hyped AI powerhouse that everyone is talking about these days ChatGPT-4🔥

Prompt

When dealing with ChatGPT (and other language models like Bing Chat), prompts are crucial, and hopefully, I didn't do a poor job here. So, I explained to the supermind what our program is expected to do and asked it to fix any bugs it can find in the code:

ChatGPT didn't notice the problems detected by Rider, but it found another one!

Wow! The machine figured out the possible values for the Rating property of the User class and noticed that it would be better to include 0 in the range of values that belong to the Beginner title! To be honest, I didn't expect any of the competitors to notice this issue, and I must admit that it's impressive!

Conclusions

So, what conclusions can we draw from this small competition?

1. First, if you write C# code for money, then go ahead and purchase a license for either Rider or for Resharper to use it with Visual Studio (if you haven't done already).

2. Visual Studio is continually improving, but it remains a few steps behind Rider / Resharper. It has always been trailing (since the first version of Resharper) and likely will be in the foreseeable future.

3. ChatGPT-4 is a beast! I've been playing around with it (not only coding and tech) since it became available, and I'm often (though not always) astonished by the results, which exceed my expectations. It's uncertain how quickly and in what manner these AI monsters will integrate into our daily work (as copilots, smart code analyzers, bug detectors, etc), but one thing is clear this is the future.

Thank you for reading, use the right dev tools and see you soon!

Cheers!