How do spiders work?

Home

Custom Search

There is something to learn about the search engines spiders: their behaviour is not logical at all!
Most of the spiders are silly and wander at random. 50% of those that find your site look for the file robots.txt (which you probably forgot to insert) and leave. It will come back a few weeks later asking for the same thing and leave again. And it will happen repeatedly for months.
You will ask yourself: Why? Well...rarely a robot looks for other pages and, if they do, they look for 1 page - sometimes 2 - each visit.
So you have no reason to be worried about robots.
It takes 6 months for them to find a sufficient number of pages.
For this reason, think about building your pages correctly and have some sites linking to yours: one day the robots will find you.

Go to any search engine, type in your URL without http://www. and you will find the list of all the pages that link to your site.
If the list is long, the robot will think that your site is very popular.
Then type in a URL of a very popular site with a content similar to yours and find out how many sites link to it: the result will demonstrate that I am right.

Search engines are nothing but links to your site.
The only problem is that you have to work hard to be listed.
So leave it, let someone else waste 10 hours a day trying to be better than others.
Stop trying to compete with big companies that can afford to spend a fortune.
Once your site gets indexed, start thinking about reciprocal links. At that point, search engines will be convinced that your site is real and receives many visitors.
At that moment you will have gained the traffic you deserve.

Often, people visiting a search engine are not expert users. Many visitors remain on your page 5 seconds then leave. It happens to everybody who uses a search engine.
Maybe because they haven't found the information they were looking for or because the site has too much graphics and it is too heavy to load.
These visitors appear on your statistics as new visitors.
Reciprocal links, on the contrary, attracts expert visitors.
As expert visitors, we mean people who are already on other sites having a content similar to yours.
These visitors have a clear idea about your site: for this reason they come and visit you and remain on your site longer.
At this point you should have no doubt about the kind of visitors you prefer.

The illogical behaviour of the robots teaches that you must have a simple navigation system on your site: a visitor, starting from your home page, must be able to find any page in no more than 2-3 clicks. Tags:spider search engine robot visitor

Bookmark this page

< Prev

[ Back ]

The ultimate search engine would basically understand everything in the world and it would always give you the right thing. And we're a long, long way from that.

Larry Page

seo melbourne

How do spiders work?

Content Links

RSS Newsfeed