C# Benchmark Programs

Benchmark. The C# language is fast.

Is it worthwhile to benchmark programming constructs? If we have a solid knowledge of how fast code executes, we can develop more efficient programs. This opens new possibilities.

Optimizations

Example. First we see the kinds of loops that Dot Net Perls benchmarks use. This program is what I use for my experiments. You will want to change "_max" depending on the code of each iteration—start smaller and push the limit up.

Note: Use the two loops here to time code. The two stopwatches are set up to print out the time.

Note 2: Run in a new C# console application. Adjust _max higher or lower based on how slow your iterations are.

Note 3: Always run in Release mode, never in VS—click the .exe yourself. Change the order of the tests.

Benchmarking Console application, 2 loops: C#

using System;
using System.Diagnostics;
using System.Runtime.CompilerServices;

class Program
{
    const int _max = 1000000;
    static void Main()
    {
	var s1 = Stopwatch.StartNew();
	for (int i = 0; i < _max; i++)
	{
	}
	s1.Stop();
	var s2 = Stopwatch.StartNew();
	for (int i = 0; i < _max; i++)
	{
	}
	s2.Stop();
	Console.WriteLine(((double)(s1.Elapsed.TotalMilliseconds * 1000000) /
	    _max).ToString("0.00 ns"));
	Console.WriteLine(((double)(s2.Elapsed.TotalMilliseconds * 1000000) /
	    _max).ToString("0.00 ns"));
	Console.Read();
    }

    static int _temp1;
    static int _temp2;

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Method1()
    {
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Method2()
    {
    }
}

Benchmarking Console application, 3 loops: C#
    This prints out milliseconds.

using System;
using System.Diagnostics;

class Program
{
    static void Main()
    {
	const int m = 1000000;
	Stopwatch s1 = Stopwatch.StartNew();
	for (int i = 0; i < m; i++)
	{
	}
	s1.Stop();
	Stopwatch s2 = Stopwatch.StartNew();
	for (int i = 0; i < m; i++)
	{
	}
	s2.Stop();
	Stopwatch s3 = Stopwatch.StartNew();
	for (int i = 0; i < m; i++)
	{
	}
	s3.Stop();
	Console.WriteLine("{0},{1},{2}",
	    s1.ElapsedMilliseconds,
	    s2.ElapsedMilliseconds,
	    s3.ElapsedMilliseconds);
	Console.Read();
    }
}

Benchmarking aspx file: ASP.NET and C#

using System;
using System.Diagnostics;
using System.Web.UI;

public partial class _Default : Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
	const int m = 1000000;
	Stopwatch s1 = Stopwatch.StartNew();
	for (int i = 0; i < m; i++)
	{
	}
	s1.Stop();
	Stopwatch s2 = Stopwatch.StartNew();
	for (int i = 0; i < m; i++)
	{
	}
	s2.Stop();
	Response.Write("Loop 1: ");
	Response.Write(s1.ElapsedMilliseconds);
	Response.Write("<br/>" +
	    "Loop 2: ");
	Response.Write(s2.ElapsedMilliseconds);
    }
}

The benchmarking code has some problems. The second block sometimes takes less time to execute due to unknown causes. Repeat and swap two loops if you are doubtful. The first program shown converts the results to nanoseconds.

For

Tip: It is usually best to report results in nanoseconds or microseconds with the number of iterations used to divide the result.

Convert NanosecondsStopwatch

Discussion. Is benchmarking important? Yes—however, it is often not important in a practical sense for your current project. It encourages you to examine your code and find out what it is really doing.

Example optimization. The Dictionary collection in the base class library is a huge optimization and reduces lookup time to a constant O(1). But developers sometimes write code that results in twice as many lookups.

TryGetValueDictionary

How to dig deeper. The JIT optimizer in .NET makes code really fast, but it can't fix everything. By benchmarking, I am able to tell what code is optimized. I gain insight into JITting.

Note: With IL Disassembler, we can see the internals of our C# code. But another way to tell that code is doing less is to time it.

IL Disassembler

And: As computer scientists, we need to understand every part of how computer languages work.

Locality of reference is important to benchmark. Things that are near are faster to access. Astronomers will tell you the speed of light. Fundamental to science is locality of reference, and this carries over into the C# language.

Locality of Reference

Summary. We looked at example code for benchmarking the C# language. Benchmarking encourages careful thinking about your code. It saves nanoseconds from your software. It also improves the depth of your understanding.

And: With benchmarking we able to grasp pipelining, file system caches, and locality of reference.

Related Links