The reading level for this article is All Levels
Your Dynamic Site is Invisible?
By Rob Thrasher
Part 1 of 3
The fuel of the search engine universe is the spider. They all have cute names like GoogleBot and they all do essentially the same thing. Some are far smarter than others, but all having one thing in common. None work at one hundred percent capacity when they attempt to catalogue dynamic, or database driven, Web sites. The spiders are automated programs that traverse the Web all day long to pull the contents of the Web into their databases. When you go to a search engine to conduct a search, the results are based upon how effective the spiders catalogue the data.
You can’t get any search engine rankings if you are not even getting spidered in the first place! There is a little known industry secret regarding dynamic Web sites, or Web-based catalogues. These sites, although a great tool, almost always break the search engine spiders. Most of the dynamic portion of the site is practically invisible to the general Web searching community.
Dynamic sites are driven by databases and code not just HTML as in the past. An example of a dynamic site is one with a catalogue driven by an internal company database providing real time and up-to-date product information to the Web browser, or searcher. These portals are highly recommended as a business tool, but they almost always erroneously communicate with the spiders.
Your site will not achieve a beneficial search engine ranking nor will it gain a Google PageRank if not efficiently spidered. Dynamic sites are notorious for their inability to get spidered efficiently. Even when a dynamic site is spidered, it is very likely spidered inefficiently. If you have a catalogue with five thousand items on the Web you should have at least ten thousand pages spidered and probably closer to fifty thousand. Proving this phenomenon is surprisingly simple and uses Google as a standard for inclusion, or lack thereof.
Coming Wednesday, May 10, 2006 : Part 2 of 3: Dynamic site = Broken spider = You’re invisible.