In response to "Yes, PHP Is Faster Than C#"

Recently there is a blog post titled Yes, PHP Is Faster Than C# that has sparked quite a conversation. I decided to run the tests mentioned in the post and found some interesting result, which I think is worth sharing.

The benchmark used here reads a file from the file system in 4 KiB chunks, and count the number of bytes with the value 1 in the file. First off, I would start by saying that I don't find this "benchmark" to be very meaningful, especially since reading files from disk is involved. There are a lot of things that can impact the file-system performance (caches, state of the disk drive, how busy the kernel is at that time), none of which is address in the test itself.
Nonetheless, the results do indicate some interesting performance characteristics that we can talk about.

Source code for the test can be found here: https://github.com/dhhoang/csharp-php-file-read

Small files

I generated the test file like this

# for this test, we will use file_size of 4 MiB as specified in the original post
base64 /dev/urandom | head -c [file_size] > test.txt

The code for the PHP (8.0) program looks something like this:

function test()
{
    $file = fopen("/path/to/test.txt", 'r');
    $counter = 0;
    $timer = microtime(true);
    while ( ! feof($file)) {
        $buffer = fgets($file, 4096);
        $counter += substr_count($buffer, '1');
    }
    $timer = microtime(true) - $timer;
    fclose($file);
    printf("counted %s 1s in %s milliseconds\n", number_format($counter), number_format($timer * 1000, 4));
}
test();

And for C#:

private static void Test()
{
    using var file = File.OpenRead("/path/to/test.txt");
    var counter = 0;
    var buffer = new byte[4096];
    var numRead = 0;
    var sw = Stopwatch.StartNew();
    while ((numRead = file.Read(buffer, 0, buffer.Length)) != 0)
    {
        counter += buffer.Take(numRead).Count((x) => x == '1');
    }
    sw.Stop();
    Console.WriteLine($"Counted {counter} 1s in {sw.ElapsedMilliseconds} milliseconds");
}
Test();

The result when running on a t3-xlarge EC2 instance is as follows (note: code is run 10 times and runtime is averaged after removing anomalies due to cold file cache)

Test-C#      53.2ms
Test-PHP     11.1ms

So the PHP code is about 5 times faster than the C# code!!! So looks like PHP really is faster than C#?

Something is definitely off here. Is .NET that slow when reading a file? Probably not. I did a simple test where I removed the "counting" part in both programs, and their performance became very similar. The blog's author claimed that the test has "very little user-land code" and mainly test the file-reading performance. I found this to be incorrect.

Now if you look closer at the 2 programs, they are very similar, except for the part where the 1 bytes are counted. PHP uses the substr_count built-in function which is very optimized, while the C# code uses LINQ. LINQ is a very convenient way to work with collections in C#, but they are also quite slow. What if we try to just count the bytes the old-fashioned way?

private static void Test_FileStream_NoLinq()
{
...
    while ((numRead = file.Read(buffer, 0, buffer.Length)) != 0)
    {
        for (var c = 0; c < numRead; c++)
        {
            if (buffer[c] == '1')
            {
                counter++;
            }
        }
    }
...
}

Our result now is (see Test-C#-NoLinq):

Test-C#             53.2ms
Test-PHP            11.1ms
Test-C#-NoLinq      6.5ms

So at this point C# is already doing much faster than before, and about twice as fast as the PHP program. This shows that the byte-counting process contributes significantly to the total run time.

So the next question is, can we do even better? When working with byte buffer, iterating through individual bytes is a pretty naive implementation. A more optimized one would be to utilize vectorization techniques such as SIMD. In fact, I would be very surprised if the substr_count function is not using vectorization. In order to test this, I created another PHP test function that iterate through the string instead of using substr_count, which would be comparable to our C# Test_FileStream_NoLinq function:

function test_manual_count()
{
    ...
    while ( ! feof($file)) {
        $buffer = fgets($file, 4096);
        $length = strlen($buffer);
        for ($i = 0; $i < $length; $i++) {
            if($buffer[$i]=='1'){
                $counter += 1;
            }
        }
    }
    ...
}

And the result (see Test-PHP-Manual-Count):

Test-C#-NoLinq          6.5ms
Test-PHP                11.1ms
Test-PHP-Manual-Count   135ms

That is painfully slow, which is why it's always a good idea to use substr_count when you need to count occurrences in a string. Unfortunately, C# doesn't not provide a built-in method with the same functionality, however it does offer a lot of primitives for implementing vectorization. I found an implementation of a SIMD-equivalent function on StackOverflow: VectorExtensions.OccurrencesOf(ReadonlySpan<byte>, byte). With this we can rewrite our counter:

private static void Test_FileStream_Vectorized()
{
...
    while ((numRead = file.Read(buffer, 0, buffer.Length)) != 0)
    {
        counter += buffer.AsSpan().Slice(0, numRead).OccurrencesOf((byte)'1');
    }
...
}

And the result (see Test-C#-Vectorization):

Test-C#-NoLinq          6.5ms
Test-C#-Vectorization   1.0ms

That is 6 times faster than manual loop and about 10x faster than PHP 😊.

Large file

For this test, I'm using an 3.2 GB Ubuntu ISO image. The result looks like this:

Test-PHP                3228.4ms
Test-PHP-Manual-Count   103966.7ms
Test-C#-NoLinq          5175.3ms
Test-C#-Vectorization   1104.7ms

Here we can clearly see how using vectorization makes things a lot faster for both languages.