.NET Framework: From Source Code to Machine Code

As mentioned in the first article of this series, in this and the next article, we will dive deep into the compiler in order to understand how the magic happens behind the curtain.

It doesn't matter how experienced you are with coding, each of us started our journey with the familiar "Hello World" application. As simple as it may appear, there is a lot going on under the hood to get your application outputs "Hello World!" on the screen.

In this blog post, we will:

• Take an in-depth look at how an application written in C# compiles to translate your source code written in high-level programming language (C#) into native code (machine code) that the machine can understand and execute.
• Learn how the Microsoft Build Engine in the .NET framework works and how it was implemented based on the bare-bones C# compiler.

The C# compilation process is a two-stage process producing compiled code in three different states as shown below.

A combination of static compilation and JIT compilation is used to compile C# to machine code as described below.

1. Developer compiles code written in C# using either the command-line or Visual Studio IDE. The compiled files are Common Intermediate Language (CIL) stored within Portable Executable (PE for 32-bit, PE+ for 64-bit) files such as .exe and .dll files for Windows. This process is referred to as static compilation since it compiles code ahead of time.

   .exe file is itself an application that can be run individually since it has an entry point (main function). .dll file is a Dynamic Link Library file that can not be run by itself since it doesn't has an entry point. Library functions in the DLL files are linked to the application at run time dynamically.

2. When a .NET program is launched, the OS invokes the CLR's just-in-time compiler, which then compiles the previously compiled code (from the step above) into machine code suitable for the platform on which it is running. This is referred to as JIT compilation which allows the CLI-compliant languages to run on a lot of platforms and compiler types.

You can build a program using either the csc.exe compiler program from a command prompt or by using Visual Studio IDE. Visual Studio uses Microsoft Build Engine (MSBuild) to load and build managed projects. It's more convenient since it knows where to find assemblies, references. etc. based on your solution and project files. MSBuild doesn't depend on Visual Studio. You can also run msbuild from the command prompt if you don't have Visual Studio IDE installed. So what is MSBuild?

What is MSBuild?

MSBuild Engine is a platform for building applications. This engine provides an XML schema for a project file that controls how the build platform processes and builds software.

MSBuild uses an XML-based project file format (.csproj, .vbproj, etc.) that allows developers to describe the items that are to be built, and also how they are to be built for different OS and configurations via the four parts of MSBuild: properties, items, tasks, and targets. We will discuss the XML-based project file format in more detail later on by looking at an example. For now, all you need to know is that this file is what MSBuild Engine uses to build.

That's it for this blog post. In the next article, we will walk through a simple console application and dive into the compiled code in order to understand how everything is wired under the hood. Stay tuned!