Well first off you’re not alone…

Back on April 17th, 2006 I was running a routine keyword check on the Info Vilesilencer site in the MSN Search Engine, and was surprised to see the site nowhere on the first page. My immediate thought was What the ? before concluding that I must have mistyped the keyphrase. When I realised that wasn’t the case I dug a little deeper issuing a site: command and my suspicions were confirmed when only 1 result was returned. MSN had totally deindexed my site.

At first I couldn’t fathom what would trigger the dropping of the site so I decided to ask on an SEO Forum that I visit regularly and soon found that I wasn’t the only one who had sites dropping. Initial speculation was that MSN had database or technology issues, however this was debunked by several people who believe that MSN uses very up-to-date search engine technology.

The discussion veered off-topic a little as Google was mentioned as using old celeron technology for its engine, and this unearthed a very interesting article that delivers some inside information about how Google Search actually works.

Another theory was given that perhaps my site was removed because it had been down at some stage during the website redesign I had done recently. It has been alleged that MSN is very unforgiving if the bot crawls your site and finds it down, and the consensus is to remove the site from the index rather than waiting till it is back up. I pondered on this, and it was true that I had an apache problem that was causing the site excessive loading times (the site would not load inside 4 minutes - it was a rogue stats application running in the background queueing jobs).

After 2 weeks the site was still nowhere to be found in MSN. The site: command still returned the same 1 result (which incidentally was the SEO Friendly Free Directory List excel), and I was beginning to worry that this was going to be a permanent removal from the index.

On week 3, since the removal, I noticed that a few more pages were re-indexed and whilst I still hadn’t regained those #1 spots for keyphrases, it was heartening to know that this was indeed a temporary problem. So why was the site removed? I can only guess at the reason why the site was removed from the index. The fact that many people were experiencing the same thing, around the same time, leads me to believe that the good people at MSN have spent time reindexing the database in preparation for the Window’s Live Beta as this had been in its initial phase of release at around the same time. However, I have no actual evidence of this, and it could be merely coincidental.

By May 15th, the domain was again number 1 for the vilesilencer keyword and I deduced that I had been close to fully re-indexed and re-ranked by that stage. Two other participants in the thread also saw their sites reindexed over the same timeframe.

So what’s the best advice I can give if your site has been recently totally de-indexed from MSN?

Be patient.

Firstly check to make sure that nothing on your site has triggered a penalty of any sort (also check Yahoo and Google to ensure this penalty hasn’t been triggered elsewhere). If your site isn’t under penalty, then my suggestion would be to be patient and wait. If after 3 weeks the bot still isn’t crawling/indexing your site again you might have a more serious problem. Personally I don’t think it’s worth getting too worried about though until that 3-4 week timeframe has elapsed, because you might be getting yourself in a knot over nothing.

If your site has recently had pages dropped from Google, then this explanation about The madness of King Google may enlighten you as to why your site is no longer Google’s friend.