ASP.NET Request.Headers

Request.Headers provides useful information.

With the data in this collection, we can decide how to render a response. And we can generate statistics about the visitors of a website. We instrument this collection here.

Example. First the data we see here is collected from the NameValueCollection on a popular website. This code example shows how to access the collection of HTTP headers in ASP.NET. It uses Page_Load and the NameValueCollection.

Example that displays Headers: C#

using System;
using System.Web.UI;
using System.Collections.Specialized;

public partial class _Default : Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
	NameValueCollection headers = base.Request.Headers;
	for (int i = 0; i < headers.Count; i++)
	{
	    string key = headers.GetKey(i);
	    string value = headers.Get(i);
	    base.Response.Write(key + " = " + value + "<br/>");
	}
    }
}

When you place the above code in a code-behind file and execute it, you will see all the headers in the Request.Headers collection. These are the headers that are analyzed. When run, it displays the page shown above.

Record values. You can add a C# code file to the App_Code folder in your ASP.NET project. This code is a static public class that will record each individual value for each key in the Headers collection.

Tip: You must call it on each Request, either in Global.asax or your Page_Load method.

Class that collects headers: C#

using System.Collections.Generic;
using System.Collections.Specialized;
using System.Text;
using System.Web;

public static class HeaderStatTool
{
    static List<HeaderData> _list = new List<HeaderData>();

    class HeaderData
    {
	public string Key { get; set; }
	public int Index { get; set; }
	public string Value { get; set; }
    };

    public static void Record(HttpRequest q)
    {
	try
	{
	    NameValueCollection headers = q.Headers;
	    if (headers != null)
	    {
		for (int i = 0; i < headers.Count; i++)
		{
		    _list.Add(new HeaderData()
		    {
			Key = headers.GetKey(i),
			Index = i,
			Value = headers.Get(i)
		    });
		}
	    }
	}
	catch
	{
	}
    }
}

The class shown above is a public static class that contains a static List of HeaderData instances. It defines a nested class called HeaderData, which contains three automatic properties.

ListStatic List

Record is a public static method that receives a HttpRequest instance from the System.Web namespace. It does not throw exceptions so won't bring down your program. The Record method enumerates through the Headers collection.

And: It records each individual header from the Request into a new HeaderData instance. It uses the collection initializer syntax.

Diagnostics. The above code will record all the header data from your web application. The next method here I show uses some logic with List and Dictionary to report the important parts of the HTTP headers sent to your application.

Property that displays collected headers: C#

public static string Diagnostics
{
    get
    {
	// 1: Store results in StringBuilder
	StringBuilder builder = new StringBuilder();
	try
	{
	    // 2: Get all names of headers
	    var nameDictionary = new Dictionary<string, bool>();
	    foreach (HeaderData data in _list)
	    {
		if (!nameDictionary.ContainsKey(data.Key))
		{
		    nameDictionary.Add(data.Key, true);
		}
	    }
	    // 3: Loop through names
	    foreach (string name in nameDictionary.Keys)
	    {
		try
		{
		    // 4: Write name and count of the name
		    var nameList = _list.FindAll(item => item.Key == name);
		    builder.AppendFormat("Header: {0} = {1} items\n",
			name,
			nameList.Count);

		    // 5: Count index frequencies
		    for (int i = 0; i < 10; i++)
		    {
			int count = nameList.FindAll(item => item.Index == i).Count;
			if (count != 0)
			{
			    builder.AppendFormat("  Index: {0} = {1} items\n",
				i,
				count);
			}
		    }
		    // 6: Get frequencies of values
		    var valueDictionary = new Dictionary<string, int>();
		    foreach (HeaderData data in nameList)
		    {
			if (valueDictionary.ContainsKey(data.Value))
			{
			    valueDictionary[data.Value]++;
			}
			else
			{
			    valueDictionary.Add(data.Value, 1);
			}
		    }
		    // 7: Write values and frequencies
		    foreach (var keyValuePair in valueDictionary)
		    {
			builder.AppendFormat("  Value: {0} = {1} count\n",
			    keyValuePair.Key,
			    keyValuePair.Value);
		    }
		}
		catch
		{
		    builder.AppendFormat("Error: {0}\n", name);
		}
	    }
	}
	catch
	{
	    builder.Append("Error\n");
	}
	// 8: Return the result string
	return builder.ToString();
    }
}

Output

Header: Host = 1730 items
  Index: 0 = 4 items
  Index: 1 = 25 items
  Index: 2 = 42 items
  Index: 3 = 44 items
  Index: 4 = 642 items
  Index: 5 = 375 items
  Index: 6 = 496 items
  Index: 7 = 94 items
  Index: 8 = 3 items
  Index: 9 = 5 items
  Value:  = 1714 count
  Value:  = 16 count

Header: Accept-Encoding = 740 items
  Index: 1 = 11 items
  Index: 2 = 265 items
  Index: 3 = 175 items
  Index: 4 = 276 items
  Index: 5 = 12 items
  Index: 6 = 1 items
  Value: gzip, deflate = 282 count
  Value: gzip,deflate,bzip2,sdch = 95 count
  Value: gzip,deflate = 227 count
  Value: deflate, gzip, x-gzip, identity, *;q=0 = 10 count
  Value: gzip, x-gzip = 10 count
  Value: gzip = 106 count
  Value: identity = 3 count
  Value: gzip, deflate, identity = 6 count
  Value: gzip,identity = 1 count

Diagnostics string property. The above code shows the Diagnostics property, which loops through and tabulates the header data instances in the List. It builds up a StringBuilder of the final report.

String PropertyStringBuilder

What it reports. The method reports each HTTP header found in ASP.NET, and prints the number of times it was encountered. Next, it prints the indexes of each HTTP header and how often each index was found.

Finally: It prints out all the values encountered and the number of times they were used.

Benefits. You can benefit from analyzing HTTP headers used with your site. For example, all 740 Accept-Encoding requests support gzip if they support deflate. Therefore deflate support is a subset of gzip support.

Cookies: You can see what cookies are used. The code will also let you see what cookies are being sent to your application.

Note: Web analytics can tell you language values, but you cannot quickly debug your site with them.

Detect bad bots. When you run a big website, you will get bad bots that scrape your content and waste your bandwidth. If you are being hammered with requests, you can use this sort of diagnostics code to help detect problems.

Detect Googlebot and MSN Bot. My statistics determined that Googlebot and MSN Bot set the From HTTP header, which you can access in ASP.NET with the Headers["From"] code. This information is likely outdated.

Note: You can then serve different content to web bots. Be careful with this though and play fairly.

Summary. We developed code to add real-time live reporting of the kinds of HTTP requests that are accessing your ASP.NET server. This helps with debugging and can lead to correctness and performance enhancements to your site.

NameValueCollection

Related Links