The Importance Of The Robots NoIndex Meta Tag, Or: What’s The Opposite of SEO?

I’m not much into SEO (search engine optimization).  It’s a booming industry for some, but I still have this crazily naive idea that good content will just naturally rise to the top of search results.  That said, there are a some basic SEO guidelines that I try to follow on my site.  When I recently discovered my site was disappearing from Google results, I learned a very important SEO lesson.

There are some pages on my site that I actually don’t want in Google search results.  For example, perhaps counter-intuitively, my blog pages:  The content there is constantly changing, so I don’t want it to appear as a search result, because it will quickly be out of date, and users will be confused when they arrive.  Instead, I want people to find the individual blog post pages where the content will remain pretty constant.  So I placed meta tags at the top of my blog pages to prevent them from being indexed:

<meta name=”robots” content=”noindex” />

Now fast forward to a few weeks ago, when I was idly going over some web site logs.  According to my logs, the most popular pages on my site are “How To” posts, and the most requested page of all time is How To Rename a Windows Service.  Out of curiosity, I ran a Google search to see how far down this one particular page would be in the results, thinking it should be pretty high if people keep finding it.

It was indeed on the first page of results… on krehbiel.blogspot.com.  (I used to crosspost there.)  But on thomaskrehbiel.com?  It wasn’t there.  Like, anywhere.  Not on result page 1, 2, or even 20.  I typed in the exact title of the page, “How To Rename A Windows Service by Thomas Krehbiel,” in quotes, and got nothing.

I checked Google Webmaster Tools and quickly found the reason.  Of the 1221 pages in my sitemap, only 58 pages were indexed.  Several days later, only 4 were indexed!  Dubya Tee Eff?

Well, I think I found the cause.  Some time ago, I made some code changes in the area of my blog that renders the <head> tags.  A colossal blunder on my part allowed <meta name=”robots” content=”noindex” /> to go on not just the blog pages, but on every single page of the site.

I’ve corrected the problem and resubmitted my sitemap; now I just have to see how long it will take to get my pages back into the index.

So here’s the lesson for anyone looking to improve their SEO skills:  Excluding your entire website from the Google index definitely does not improve your search result rankings. :)  But seriously, make sure the meta robots tag is correct.  (Also, it would probably help to check your logs and statistics more than once or twice a year.)

(However, if you want to make a relatively private site, robots noindex is a very effective solution.)

Development Software I Use Now

I’ve setup a new development PC, so I thought it was time once again to review the software I normally install and use for development (I last did this in December 2007).

I find that most of these kinds of lists are pointless, but every now and then I run across a handy tool I’ve never seen before, so in the hope that someone else might find something useful, here’s my current list of PHP and .NET development essentials.

Note that I get along just fine without Resharper or any ORM, Mocking or Dependency Injection frameworks.  That’s just how I roll.

* I don’t consider these essential, but they are instructive.

How to Rename a Windows Service

I came across a situation where I wanted to rename a Windows service that had already been installed.  (In this case I wasn’t able to rebuild the application to alter the service installation properties, which would of course be the ideal solution.)  Google was spectacularly unhelpful, so this is how you do it:

Open RegEdit.

Navigate to HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServices.

Find the key for the service you want to change and simply rename it.

RegEdit screenshot

A reboot was necessary for the change to take effect in the Services snap-in.

You may also encounter a sub-key called DisplayName that you can change.  This is not the same as the key above.  When using “net start” and “net stop,” for example, you need to refer to the key name above.

DisplayName screenshot

I did this on Windows XP and Windows Server 2003; presumably it will also work in Vista, etc.

STANDARD REGEDIT DISCLAIMER:  Do not attempt this unless you know what you’re doing.  The consequences of a mistake could be disastrous.

Get Microsoft Virtual PC

I’ve heard about “virtualization” for years, but I never thought it was anything I needed to worry about.  I just thought it was something for network engineers to tinker with in big, expensive data centers.  It turns out there are plenty of handy uses for virtualization, using Microsoft’s Virtual PC, in your everyday computing life.

For example, if you want to test the latest build of your software installation package, you can fire up a Virtual PC and run your installer on it.  If it breaks, you simply rollback the hard drive changes and you’re back to a fresh hard drive.  You can test on any version of Windows without having to setup any hardware.  It’s way simpler and faster and cooler than keeping a spare dinosaur computer around with Norton Ghost images.

Perhaps you have to work in a restrictive network where you don’t have access to any other computers for testing.  Fire up a Virtual PC and create whatever test environment you want.  You can connect it to the network or not, depending on your needs.  If you break something, you can roll it back to the starting point in seconds.

Or maybe you want to try out some shareware programs but you’re not sure if they’ll install a bunch of spyware or adware.  (You know as well as I do that once you install Windows software, you usually can’t get rid of it without reformatting.)  So fire up a Virtual PC and install it there first.  The virtual environment is completely isolated, so there’s no danger of getting any infections if the software’s bad.  You can try it out for a while to see if you like it before committing it to your main workstation.

How’s it work?  I have no idea.  I can only assume it’s “emulating” a PC, in the same way that people have always been writing emulators for older computer systems.  Normally emulators require considerably faster hardware for the emulation to run at a normal speed.  Virtual PC, however, somehow manages to run well even on the same hardware.  It’s noticeably slower, of course, but certainly not unusable.  It’s about like running an RDP session over a slow connection.

So go ahead and download Virtual PC 2007.  It’s even free.

When It’s Okay To Use C# 3.0 var

I ragged a little bit on “var” a while ago, but there are some cases where I think it’s a handy shortcut.  Those being when it’s abundantly clear what the underlying type is, like when creating new objects.  For example, it makes perfect sense to change this:

CustomerDataContext db = new CustomerDataContext();

To this:

var db1 = new CustomerDataContext();

It’s clear to me at a glance what’s going on when I’m reading it, and there’s no needless redundancy.

(By the way, you can’t use “var” in a class-level declaration, only in local declarations.  I don’t know why, either… it feels like a completely arbitrary restriction.)

The problem area for “var” is when you use it with method calls, ie. changing something like this:

FileInfo[] dirs = GetFiles();

To this:

var dirs1 = GetFiles();

You suspect it’s returning a collection of files by looking at the method name, but you don’t know for sure and you don’t know exactly what kind of collection.  It could be an Array or a List<T> or an ArrayList or an IEnumerable or an IEnumerable<T>.  In that situation, I would probably discourage people — especially less experienced people* who tend not to write descriptive variable and method names – from using a var shortcut.  (You can always hover the mouse over GetFiles to see the method signature and return type, but that’s an extra step.)

I also wonder about using var as the return type for LINQ queries.  Sure, we all know from experience that you’re getting back an IEnumerable<T> collection of one type or another, but what about someone who’s just out of college*, looking at a LINQ query for the first time?  That person probably won’t understand the LINQ query syntax or “var” (since I doubt it is taught in any school), but I would think they’d know what to do with an IEnumerable.

Maybe I’m being a little too alarmist there.  I guess using LINQ queries at all should imply a certain level of knowledge to work with the source code.

* The older I get, the more I find that a lot of my code crafting efforts go toward making the codebase ready for less experienced programmers to work with.  Which basically means:  Simplify, simplify, simplify.

CookieContainer, HttpWebRequest and Secure Cookies

I ran across a troublesome problem that took several evenings to debug.  I couldn’t find a solution with Google so maybe somebody else will benefit from this.

My app sent an HTTP GET to a secure site using an HttpWebRequest and a CookieContainer (to capture the site’s cookies).  Two cookies were returned in the response header, one of which was marked by the server as “Secure.”  Both cookies were in the CookieContainer (the Count was 2 and they were both returned by GetCookies(url)).  However, I subsequently sent an HTTP POST using the same CookieContainer to the same site (still https), but HttpWebRequest did not place the secure cookie into the POST header.

This might be by design, a misunderstanding on my part, or a problem with the site’s cookies, but in any case it was not working.  To get around the problem, I had to manually add the missing cookie to the CookieContainer before calling HttpWebRequest.GetResponse.  Only after doing that was the cookie added to the outgoing request header and subsequently sent to the server.  Something like this:

CookieContainer cookieContainer = new CookieContainer();

 

HttpWebResponse response = DoHttpGet( cookieContainer );

 

// Manually add the missing cookie.

Cookie cookie = new Cookie( “Name”, response.Cookies[“Name”].Value );

cookie.Secure = true;

cookie.Domain = “www.domain.com”;

cookieContainer.Add( cookie );

 

DoHttpPost( cookieContainer );

Alternatively, I found that this worked also:

HttpWebResponse response = DoHttpGet( cookieContainer );

// Alternative to the above.

cookieContainer.Add( new Uri( “https://www.domain.com” ), response.Cookies );

DoHttpPost( cookieContainer );

However, a plain cookieContainer.Add( response.Cookies) — a frequently-referenced Google solution – did not work.

HttpWebResponse response = DoHttpGet( cookieContainer );

// This did not work.

cookieContainer.Add( response.Cookies );

DoHttpPost( cookieContainer );

By the way, Fiddler was invaluable in debugging this issue.

Disabling Pluralization in Visual Studio

I noticed with some concern that the Visual Studio 2008 designer puts an “s” on the end of table names when it builds LINQ to SQL classes.  If you want to disable that, open Tools/Options, go to the Database Tools O/R Designer page, and change “Pluralization of names” option to false.

Note that if you have already generated pluralized classes, you have to force the designer to regenerate the classes by actually changing the names to some temporary value, build, then change them to what you really want and rebuild again.  Otherwise, if you leave the name the same, it assumes it doesn’t need to regenerate the classes.

New Collection vs. Collection.Clear

Quick, which one of the following implementations is faster to use in a time-critical section of code:

Implementation 1:

   25 for( int i = 0; i <</SPAN> iterations; i++ )

   26 {

   27     List<</SPAN>int> myList = new List<</SPAN>int>();

   28     for( int j = 0; j <</SPAN> 1000; j++ ) myList.Add( j );

   29 }

Implementation 2:

   34 List<</SPAN>int> myList = new List<</SPAN>int>();

   35 for( int i = 0; i <</SPAN> iterations; i++ )

   36 {

   37     myList.Clear();

   38     for( int j = 0; j <</SPAN> 1000; j++ ) myList.Add( j );

   39 }

Answer:  The second one, by about 33%.

    1 using System;

    2 using System.Collections.Generic;

    3 using System.Diagnostics;

    4 using System.Text;

    5 

    6 namespace CollectionSpeedTest

    7 {

    8     class Program

    9     {

   10         static void Main( string[] args )

   11         {

   12             long time1 = Stopwatch.GetTimestamp();

   13             Method1( 1000000 );

   14             long time2 = Stopwatch.GetTimestamp();

   15             Method2( 1000000 );

   16             long time3 = Stopwatch.GetTimestamp();

   17 

   18             Console.WriteLine( “Method1: {0}”, time2 time1 );

   19             Console.WriteLine( “Method2: {0}”, time3 time2 );

   20             Console.ReadLine();

   21         }

   22 

   23         public static void Method1( int iterations )

   24         {

   25             for( int i = 0; i <</SPAN> iterations; i++ )

   26             {

   27                 List<</SPAN>int> myList = new List<</SPAN>int>();

   28                 for( int j = 0; j <</SPAN> 1000; j++ ) myList.Add( j );

   29             }

   30         }

   31 

   32         public static void Method2( int iterations )

   33         {

   34             List<</SPAN>int> myList = new List<</SPAN>int>();

   35             for( int i = 0; i <</SPAN> iterations; i++ )

   36             {

   37                 myList.Clear();

   38                 for( int j = 0; j <</SPAN> 1000; j++ ) myList.Add( j );

   39             }

   40         }

   41     }

   42 }

Results:

Method1: 20962096146
Method2: 13987606779

Finally != Inevitable

For future reference:  I had always heard that there were cases where finally blocks were not run.  The other day I actually witnessed one of those situations:  In a .NET console app, if you hit CTRL-C to exit the application, finally blocks are not executed.  Fortunately garbage collection will still handle standard cleanup of objects for you (at least I assume so).  But if you need to do something special you have to use the System.Console.CancelKeyPress event, like so:

System.Console.CancelKeyPress += delegate { Cleanup(); };

Be careful because it runs in a separate thread — the main thread is not interrupted and won’t actually stop until the CancelKeyPress handler returns.

Unfortunately, I found no way to handle the situation where the user clicks the close button (the red X) on the console window.  It skips both CancelKeyPress and all the finally blocks.

NAnt MissingManifestResourceException Tip

With NAnt 0.86 Beta 1, I found that I couldn’t use the <resources> element to include *.resx files in my <csc> task. For example, this didn’t work:

<csc target=winexe output=${bindir}\UvNotes.exe debug=${debug} win32icon=UvNotes.ico >

<resources>

<include name=*.resx />

</resources>

<sources>

<include name=*.cs />

</sources>

</csc>

The program wouldn’t run, failing with a MissingManifestResourceException.

Initially I tried to do this:

<resgen todir=${objdir}>

<resources>

<include name=*.resx />

</resources>

</resgen>

<csc target=winexe output=${bindir}\UvNotes.exe debug=${debug} win32icon=UvNotes.ico >

<resources>

<include name=${objdir}\*.resources />

</resources>

<sources>

<include name=*.cs />

</sources>

</csc>

But that also failed because it was generating resource files named eg. ”NoteForm.resources.”

To finally fix it, I ended up doing this:

<resgen input=NoteForm.resx output=${objdir}\UvNotes.NoteForm.resources />

<csc target=winexe output=${bindir}\UvNotes.exe debug=${debug} win32icon=UvNotes.ico >

<resources>

<include name=${objdir}\*.resources />

</resources>

<sources>

<include name=*.cs />

</sources>

</csc>

That generated resource files named eg. “UvNotes.NoteForm.resources” which was apparently required for it to work properly.