Did you know that search engine spiders frequently skip many of the interior pages when they crawl larger sites? Especially if those pages are more than one level down from the root directory of a site, the spiders often ignore them.
One traditional method of overcoming this has been to submit every page of a site to the major search engines, but that can be terribly time consuming if you do it by hand, or expensive if you employ a service to do it for you.
Bloggers and webmasters are creative folks, though, and some have come up with a new strategy to use blogs to get a website fully spidered. As far as I know, this strategy was invented, or at least has been popularized by, a fellow by the name of Rick Butts. He calls it the “Blog and Ping” strategy.
Frankly, I find this a little controversial, as it suggests that you create a blog for the sole purpose of getting another website indexed by the search engines. On the other hand, if you have a good reason to have a blog, and can “deep link” it to your main website with legitimate posts, then this can be a great strategy to get some of those hidden interior pages indexed.
Here’s how it works. You establish a blog whose theme is relevant to the theme of your main website. I guess it doesn’t absolutely have to be relevant, but that would sure help, as I see it. This is a linking and SEO strategy, and search engines show preference for relevant links.
Anyway, once you have your blog set up, you activate the RSS feed, and get it listed in Yahoo Feeds, Google Feeds, and several other RSS directories. The simplest way to do that is to add it to your own accounts!
In addition, you need to make sure that you’re set up to “ping” the blog directories every time you post a new entry to your blog. Personally, I just use PingOmatic and set it up to be pinged automatically from WordPress.
The easiest way to implement this strategy is to set this blog up at Blogger or WordPress. You can host it anywhere, but hosting at WordPress.Com or Blogger.com pretty much ensures quick and repeated visits from the GoogleBot (Google now owns Blogger.Com).
Some people prefer to just set the blog up as a new section on an existing web site, which is fine, too, and has the advantage of making your main site content grow and expand.
OK, once you’re set up, you’re ready to “blog and ping”. Post a couple of entries to the new blog to get the ball rolling. If you post daily for a week or two, you can be reasonably sure that both Google and Yahoo will have spidered your blog.
Don’t overdo it. At one time, some advocated posting and pinging multiple times per day, with any content whatsoever, but the search engines got wise to that in a hurry, and started banning blogs that were updated too frequently. The blog directories also got wise, and “blacklisted” pings from blogs that appeared to be spamming them.
Stick with one or two posts per day, especially at the beginning, and then it’s time to rock and roll after a week or two.
From this point on, you want your blog entries to contain “deep links” to the interior pages of the website you’re trying to get spidered. A “deep link” simply refers to a link that goes straight to a specific page, rather than to the main page, which is the traditional target for links. Deep links target the hard-to-find interior pages.
From now on, whenever you add a new post to your blog, slip in as many of these “deep links” as you reasonably can. In my opinion, the blog entries must be legitimate, and serve some useful purpose for your human visitors. That shouldn’t be that difficult, however, if the theme of your blog is related to the theme of your website.
Some people think that you can still implement this strategy without bothering about the content, relevance, or frequency of your posts, but I, personally, am opposed to that.
I feel it amounts to sp*mming the search engines. To each his own, I guess …
So what happens when you “blog and ping”? Well, as we all know, search engines love blogs because of their constantly freshened content, and the fact they are full of links. So the bots come crawling to your blog, and they follow the links that they find … and follow the next set of links that they find at the pages you send them, to, etc, etc, until they get tired of it.
Since the bots are visiting your blog frequently, they’re also visiting most, if not all of the pages of your main site frequently, and you, as a result, get your whole site indexed much more quickly than you normally would.
NOTE: this does NOT get your pages ranked any better than they normally would! It just gets them indexed quickly. It’s up to YOU to optimize the pages of your website, and get additional links to and from them, to get the high page rank we all desire.
The strategy does, however, get your site fully indexed, and gives you a chance to get lots more of your interior pages listed in the search results much more quickly than usual.
As I said several times above, I personally don’t believe you should create and post to a blog just to get your main website indexed. On the other hand, if you have a relevant blog, and have a legitimate reason for linking some of the posts to the interior pages of your site, this can be a very useful strategy.
Happy Blogging!
