IIS7 Hijacking my custom error pages

Custom error pages replaced by IIS?

Development Machine W7, Production IIS7 Windows 2008

Just had a sticky problem crop up in production. Custom error pages that used to work no longer work, I don’t know when but sometime our custom ASP.NET error pages have been replaced by stock IIS error pages.

How we use custom error pages

If a website user or search bot attempts to access a product page and the product no longer exists, we send them to a custom error page. The page suggests they try another product or search for alternatives. This page sends back a http 404 status so that the search engines know to drop the accessed URL from the index. It also causes our Google Mini to drop the page from its database too. If the user navigates to another url that has never had content they get a default 404 page returned.

Lets have a look

So you have a great little custom error page like this;

Protected Sub Page_Load1(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
        'Ensure a 404 not found sent for SEO purposes
        Response.StatusCode = 404
        Response.TrySkipIisCustomErrors = True
    End Sub

However even browsing direct to the page you only get,

IIS7 404 default error page

This is not your crafted custom page, instead this is the IIS7 page kicking in. IIS has put itself in the pipeline to the end user, replacing its own page with in place of the custom page.

IIS custom page configuration

The configuration for this is found in the site Error Pages section;

IIS7 Management Link to Error Pages

If you go into this menu you see all the custom error pages set up in IIS, now select the edit feature settings link on the right hand side;

IIS7 Error Pages Configuration

This is where the behaviour is set and currently this is set to show detail errors for users local to the machine but custom errors for people elsewhere. Note this is not ASP.NET, although the idea is the same, this is above ASP.NET, it is the handling IIS does for these pages.

You do not want normal users to see the detailed errors as it can give away important information about the configuration of your server thus the setting shown is fine and looking at it one would guess it would show our custom pages as we saw by the returned IIS 404 page this is not the case.

IIS7 Custom Errors Settings

So why do we not see our custom pages? Well back in the page code behind shot earlier is the key, you must set

Response.TrySkipIisCustomErrors = True

This tells IIS to get its nose out of what we’re up to and let us get on with it. Trying to browse the custom error page, now we get our custom error page retuned as would be expected in the first case.

Lesson learnt

The real nasty bit of this is that our development machines do not exhibit this behaviour, everything looks fine there. As Rick Strahl says it is another good case for using the staging servers with the production environment so that you test everything.

Other links:

Rick Strahl's Web Log has an almost identical post here

What to expect from IIS7 custom error modulethis link was the one that taught me what I needed to solve the problem.

response.TransmitFile bad coding day

File Downloads from ASP.NET

All I had to do was serve the file as a download page to the customer, there the pain began.

If providing file downloads, it is important to buffer stream the file to the user to manage the server memory. If many users start downloading your files together and you are not buffering the file, then the whole will be held in memory multiple times thus causing memory exhaustion. Buffering means only a few bytes of each downloading file is held in memory at once.


In .NET 2.0 a great new method was added to the framework, “response.TransmitFile”. This method takes out a load of code that we used to have to code whenever we wanted to provide a file to the customer for download. It streams the file to the user and handles the buffering of the file. Look at the following simple code.

response.ContentType = "application/x-msexcel"
response.AppendHeader("Content-Disposition", "attachment; filename=PriceList.xlsx")


Now compare that to this version where no buffering is ocurring, the whole file is read to memory then sent to browser  Protect files for download - Code Project 

'This is the function that you have to use
Private Function downloadfile(ByVal strFile As String)
    Dim fs As FileStream
    Dim strContentType As String
    ' This is the important part, because you going to use the local path 
    'to get the file
    Dim strPath = Me.Server.MapPath( _
        System.Configuration.ConfigurationSettings.AppSettings("uploadDirectory")) & "\"
    Dim strFileName As String = strFile
    fs = File.Open(strPath & strFileName, FileMode.Open)
    Dim bytBytes(fs.Length) As Byte
    fs.Read(bytBytes, 0, fs.Length)
    Response.AddHeader("Content-disposition","attachment; filename=" & strFileName)
    Response.ContentType = "application/octet-stream"
    Return True
End Function


The problem

I got the error below when I implemented my download page;

The file you are trying to open, xxxxxx is in a different format 
than specified by the file extension. Verify that the file is not corrupted 
and is from a trusted source before opening the file. 
Do you want to open the file now?

I could not open the excel file successfully. I opened the downloaded file in textpad to look at what was happening inside, but as office stores its files in zips this didn’t reveal anything immediately to my eye. Instead I tried using a text file to download and that revealed the issue that lead to me finding my mistake.

The downloaded file exhibited corruption, and thus I started looking for issues that could  corrupt the file. These included tracing being switched on for the site, IIS settings, wondering if it has something to do with my new Windows 7 machine etc. After several hours from starting what should have been a twenty minute job, I discovered my error. See my erroneous below;

My error

Dim StrFilename = Files(0) 
response.ContentType = "application/x-msexcel"
response.AppendHeader("Content-Disposition", "attachment; filename=PriceList.xlsx") 
response.TransmitFile(StrFilename, IO.FileMode.Open, IO.FileAccess.Read)

See what I’d done – I didn’t for ages. As I had taken some code from another streaming download that I wrote in .NET 1.1 I copied the filename and file opening line. Only I also copied the io.FileMode.Open, IO.FileAccess.Read, as it happens there is an overload for TransmitFile that takes and offset in the file and length. Of course the enumeration for IO.FileMode.Open resolves to an integer so no compile error or runtime error, instead my file was truncated to virtually nothing!

On getting rid of these two parameters in all worked great! A number of hours wasted eliminating issues down to the line of code that were wrong. Funny how you can create masses of functionality and code one day and another get bogged down by a stupid error like this!

Windows Media Services & Windows 2008

Win2008 I missed something important about Windows Media Services WMS in Windows 2008. A lot of the features that were previously only available in Windows 2003 datacenter or Windows 2003 Enterprise editions are standard in Windows 2008! Yipee and about time!

The inability to write custom authentication plugins except for the advanced windows versions prevented an old project from actually being viable. Now custom authentication plugins will work for me.

The new 2008 version is reportedly much more efficient and up to some serious traffic and has some other very relevant to features to some work I cant talk about;


Google Mini images in results ASP.NET

Providing search for our ASP.NET site has been left to an implementation of the Google Mini. This choice was for speed of set up, user familiarity with result sets generated and the fact that Microsoft Search was not released when the decision was made. The Google Mini is a 1U high rack mount server hardware supplied by Google. It provides a web browser administrator interface and lets you integrate it into your site in a few different ways. We fire requests to it and get an XML results file back that we then manipulate to produce a good search experience. We use the GSALib port of the gsa-japi Jarva library.

Now we have it up and running it has now become time to start tinkering a bit to try and get better results. The first challenge was getting product image thumbnails shown next to the search results from the Google Mini. This actually was very simple to achieve. The Mini understands meta tags in your html. If you put the following into the <head></head> section of the product pages:
<meta name="tw-prodimg" content="48350_01.jpg" />
Then then when the mini indexes your pages, this tag together with any others present, will be stored as a collection against that page’s result. You must explicitly ask for the meta tags to be included in the XML results from the Mini if you wish to consume them in the resulting XML from the search appliance.

   1: Dim objQuery As New GSA.Query
   2: Dim collections(0) As String
   3: collections(0) = _searchSiteCollection
   4: objQuery.setSiteCollections(collections)
   5: objQuery.setFrontend(_searchFrontEnd)
   6: objQuery.setOutputFormat(GSALib.Constants.Output.XML_NO_DTD)
   7: objQuery.setOutputEncoding(Constants.Encoding.UTF8)
   8: objQuery.setAccess(Constants.Access.PUBLIC)
   9: objQuery.setScrollAhead(CInt(_searchStartPageIndex))
  10: objQuery.setMaxResults(_searchPageSize)
  11: objQuery.setFilter(_searchFilter)
  12: 'Set set FetchMataFields=* to get all meta tags associated with page result
  13: Dim o As String() = {"*"}
  14: objQuery.setFetchMetaFields(o)


As you can see a call to setFetchMetaFields has been passed “*” this means return all meta tags from this pages result. You may if you prefer pass a string array of meta tags you are interested in seeing to reduce the returned tags. You may also use meta tags for filtering result sets by meta tag, but not part of this discussion.

Now  the results will include an XML node <MT> that contains the meta tags, this is exposed through the GSA library as a string collection hanging off the search page result it is associated with. Thus we can now show the image on the page using data binding in our repeater control, thus:

<asp:HyperLink ID="sImgLnk" NavigateUrl='<%# Server.HtmlDecode(eval("Url")) %>'
  runat="server" meta:resourcekey="HypImageResource2"></asp:HyperLink>:HyperLink>
   1: Protected Sub repeaterProductResults_ItemDataBound(ByVal sender As Object, ByVal e As System.Web.UI.WebControls.RepeaterItemEventArgs) Handles repeaterProductResults.ItemDataBound
   2:     If (e.Item.ItemType = ListItemType.Item) Or (e.Item.ItemType = ListItemType.AlternatingItem) Then
   3:         Dim SearchResult As GSA.Result = CType(e.Item.DataItem, GSA.Result)
   4:         If SearchResult.Metas.Contains("tw-ItemImg") Then
   5:             DirectCast(e.Item.FindControl("sImgLnk"), HyperLink).ImageUrl = ConfigurationManager.AppSettings("PathToProductThumbs").ToString & SearchResult.Metas("can-ItemImg")
   6:         Else
   7:             DirectCast(e.Item.FindControl("sImgLnk"), HyperLink).ImageUrl = ConfigurationManager.AppSettings("PathToProductThumbs").ToString & "noimage.gif"
   8:         End If
   9:     End If
  10: End Sub


Thus you now have item images against the items. You can obviously expand this so that all your pages could have a “searchimage” meta tag so that news items or other content could all have individual thumbs.