Developing games on and for Mac and Linux

The case for multi platform game development.

Introduction

In my current organization we have come to a point where we will have to test our belief in our previous choices. Up until now we have said “We will support Windows, Mac and Linux as long as it does not take up too much of our time”. Now things are heating up and we need to prioritize where we allocate our resources.

In this article I will drill into the different aspects of multi platform development and multi platform development environments for personal computers. I will try to convince you that there is a case for multi platform development and if you can afford it, and have the skills and perseverance to follow through, there is much to gain.

The below arguments all assume planned support for the two biggest console manufacturers on the market, Sony Playstation and Microsoft XBOX.

The costs

When developing an application for multiple platforms there are different aspects that will incur costs. One way to slice it is development, quality verification (testing) and distribution. Depending on the platform and choices made, they may all have equal impact on total cost.

Development

This is where extra costs arise from taking into consideration hardware and certification requirements from different vendors or platform owners.

As for pure feature development, much of this should be solved for us by the chosen game engine, in our case Unreal Engine. We did indeed choose Unreal Engine much because of its wide platform support. This is a truth with some modifications as the platforms are far from on par feature wise in the engine. Due to this, we often stumble on issues on one platform even though it worked on another platform. Up until now most differences have been related to rendering and video codecs. This is not surprising as it is key things that platform owners try to differentiate themselves on. I.e. Windows, XBOX -> DirectX, Linux -> Vulkan, Mac -> Metal, etc. and video codecs that are proprietary and sometimes hardware accelerated with special purpose on board circuits.

On top of pure technical development costs, most platforms have certification requirements that the developer needs to pass to be allowed to release the game on that platform. This could be certain ways menus are used, things that are not allowed to take too long, etc. The effect of this is that many features need special treatment and development for each platform. And this leads into the next section where we need to verify all these features.

Quality verification

Simply put, all versions and combinations of features of the game need to be tested. The more versions, the more testing and hardware is needed. With cloud services available and making automated testing a priority early on, we can minimize the testing effort but there is no getting around that each platform and configuration needs to be tested.

Worth noting is that PC-platforms are disproportionately expensive to test due to the number of hardware configurations. As an example, the Playstation 4 only exists in a couple of different hardware configurations so testing them all becomes possible. A PC could be put together with any compatible hardware making the permutations that need to be tested almost infinite.

Distribution

This one is pretty straightforward. It’s the costs related to getting the game into the hands of the players. Every platform has its own quirks and associated cost (for instance, the cost of generating a code for a digital download, etc.) but generally speaking, the biggest cost is the licensing fee. I.e. How big of a cut does Steam take on PC or Sony on Playstation. This cost does not increase with the number of platforms but it may vary depending on which platforms are chosen.

The business case

Looking at mostly game development in this article, the interesting data point is what is commonly referred to as install base. It is a simple to understand metric that means essentially: “If you build a game for this platform, how many potential customers do you have on that platform”. As we can see in the diagram, Linux accounts for only 0.5% of the potential install base and Mac only 1%. If it goes really well and the game reaches a really wide audience, say 100 million players, then that means potentially 500 thousand and 1 million players respectively for Linux and Mac.

There is an argument to be made for different kinds of games and different audiences. A game that runs on cheaper hardware with less resources and is free to play could potentially have a larger install base on for instance Linux that is a free operating system. This is, however, only speculation.

So, is the case closed? There is no point building for Linux and Mac? Not quite, read on.

The technical reasons

Compiler

If we group the platforms in a slightly different way, namely on compiler, we see that MSVC C++ compiler used by the Microsoft platforms and Clang C++ compiler used by all the others are pretty equal. Taking into account the dedicated servers that all run on Linux regardless of client platform, we can consider them roughly equal in terms of numbers. This is not interesting because it gives us any relief in development costs but it is interesting because our game code always needs to compile on MSVC and Clang C++ compilers regardless of if we chose to support Mac or Linux. So additional platforms do not add much cost in this regard.

Rendering API

Although the compiler differs depending on which platform you are compiling for and on, much of the technical complexity should be abstracted by Unreal Engine. That is, before Unreal does a release, they make sure everything is working on the supported platforms. So in theory, when we take that release, everything should be working. It is however very hard for Epic (who builds and maintains Unreal Engine) to have test cases for every feature in all rendering API’s. Our experience from doing one major, 3 minor and 2 hot-fix upgrades on the game while in development and while being live, tells us that a lot of problems arise from differences in the rendering API’s and artifacts/errors due to that. What this means is that if the game is working nicely, and we do an engine upgrade, there is no guarantee that the game will run on all platforms after the upgrade if they did it before the upgrade. The more rendering API’s we support, the more special treatment we need for each platform. This is where it gets really bad for Mac and Linux. Since DirectX is a proprietary rendering API for Microsoft platforms and Apple stubbornly wants their own (Metal), Linux needs to use the open (and still fairly young) rendering API Vulkan. We end up with the following graph if we look at it from a Rendering API perspective:

Since we technically do not need rendering on the game servers (that run on Linux) and Mac is just a small percentage of the install base, it becomes difficult to motivate the increased development effort to maintain the game on Mac and Linux.

The practical implications

Currently, 15% of our studio uses Mac, 40% of the game engineers prefer Linux. Among game engineers it is more common with Linux and among cloud engineering, art and other departments Mac is more popular. What this means is that if we do not support Mac, a significant part of the studio will need other computers/consoles to play the game if we decide not to support Mac. A large percentage of the game engineers will have to develop and/or test the game on another platform than the one they feel more comfortable with.

I’ve worked at Windows only game development studios that only supported Windows PC clients but ran all servers on Linux. What ended up happening in a studio with several hundred developers was that one developer used Linux to natively debug and profile the server build effectively, many others used remote debugging from their Windows machine to track down specific issues. What happened in practice was that the Linux build, vis-a-vie the server build, was neglected due to the hassle to even run it. The low hanging fruit can usually be fixed and optimized on a Windows version of the build but if you want to get really fast or really stable (finding those intermittent bugs) then you need to be natively working on the platform.

Another aspect that is often overlooked is the importance of the engineering team learning early to consider multiple platforms. As mentioned above, automatic testing becomes important right from the start since you do not have time to test every platform manually. The consequence of this is that the organization makes the multi-platform investments needed eventually directly and can start leveraging benefits right away.

Technically, when forced to consider multiple platforms, engineers learn early that code needs to be modular and have minimal dependencies. The modularity is due to being able to make platform specific implementations and minimal dependencies is due to external dependencies seldom supporting all the platforms you need them to. Both of the above will require more senior developers to pull off.

Social responsibility and employer branding

A well established practice in game development, and software development in general, is exclusive content. What this means is that platform owners ensure, through any means possible, that some game, applications or content is available only on their platform.

After a while, consumers learn that their desired content is available on a specific platform and that platform grows. Next step is that some game studios are doing exactly the above calculation and realize the install base is too small on some platforms so they do not develop their game for it. The circle of platform lock-in is now complete when the platform owners do not have to pay for exclusive content anymore since the other platforms are too small to motivate developing games for them. Since we cannot see the future, this becomes a chicken and egg problem. Are the Mac and Linux install bases so small due to missing content or are they missing content because the install base is so small?

The socially responsible thing to do, all other things equal, is to develop the game to be accessible to as many people as possible. Making the game accessible to for instance color blind people (~4%) is something that we take for granted, but making the game accessible to Linux and Mac players (~1.5%) is seldom talked about in relation to accessibility.

Making a game at the level of ambition that GOALS is trying to achieve, takes a lot of good developers. Finding these developers is not easy but one good way of attracting them may be letting them work on their preferred operating system. According to Stack Overflow developer survey of 2022, the distribution between the operating systems for professionals is a bit different than the one for gamers:

State of the art

If you decide to go down this road there are some fantastic tools, applications and libraries who's makers share the vision of a multi platform gaming ecosystem:

Game engine

Unreal Engine: https://www.unrealengine.com/. Open Source and multi platform game engine.

Version tracking

Plastic SCM: https://www.plasticscm.com/. Not only multi platform but specifically built for the needs of a game studio.

Integrated Development Environment

Rider: https://www.jetbrains.com/rider/. An IDE with a large ecosystem of plugins and specific Unreal Engine support.

Engine binary distribution

Gamecure: https://github.com/goalsgame/gamecure. Our own cross platform game engine distribution application that we open sourced. Read more here: https://dev.to/goals/collaboration-on-large-teams-with-gamecure-1dfb.

Engine upgrades

UEIMPORTER: https://github.com/goalsgame/ueimporter. Open sourced and in house built tool set for upgrading Unreal Engine when under Plastic SCM version control.

Code generation and network serialization

Protocol Buffers: https://developers.google.com/protocol-buffers

Delta patching

golongtail: https://github.com/DanEngelbrecht/golongtail
Cross platform library for delta patching. Used by our end user game launcher and Gamecure (mentioned above)

Summary

On the surface, it looks like an easy decision. The numbers just don't add up. Drilling a bit deeper, we realize there is much to gain from sticking to multiple platforms right from the start and having come such a long way as GOALS, and having all infrastructure in place, my conclusion is that there is no question what is better and what is desirable, the question is what the organization can afford and what the people in the organization can manage.

References

• https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam
• https://www.statista.com/statistics/1260264/global-gaming-console-installed-base/
• https://survey.stackoverflow.co/2022/

Title image credit: Midjourney