Home

Katz Tales Blog


Books

Logomania: Fate & Fortune

FREE! Katz Tales:
How Boris The Cat
Was Reunited With His Family

Blackmail Bride An ebook romance

Katz Tales:
Living Under The Velvet Paw

Logomania: Where Common Phrases Come From And
How To Use Them


Beginner's Guides

Netiquette

Effective Email

Understanding Search Engines, Directories, Meta Search Engines
and Portals

Basic Guide To Internet Research

Colour Psychology


Contact

Understanding Search Engines, Directories, Meta Search Engines and Portals

This is a chapter from my book Instant Genius: Internet Research. I wrote this under the pen name Maria O'Daniel in 2001. Although the information below is more or less a copy of that chapter, I have updated it a little. This version was put up on 1 November 2009.

Understanding Search Engines, Directories, Meta Search Engines and Portals

In order to search efficiently, it's important to be able to distinguish between the types of tools you can use to search for information and how they work.

There are so many search services about, that it's hard to discover which is best. Each search service shares the same two basic problems.  First, the amount of new web pages being created far outstrips the speed at which they can be indexed. Second, web pages are not static.  Constant updates, upgrading and changes are made to existing resources which means that there is no fixed "known territory".

How big is the Internet?  In 2010 Google announced it had indexed 1 trillion pages.  And the Net is growing every day!

In August 2009 Search Engine Watch reported that there were 10,812,734,000 online searches.  Google got 6,986,580,000 or 64% of these.  Yahoo got 1,726,060,000 or 16%.  MSN/WindowsLive/Bing got 1,156,415,000 or just under 11%.  Other big boys were AOL, Ask.com, My Web, Com Cast, Yellow Pages, Nex tag and Local.com.

However, before you decide Google is the only search service worth using because it's the biggest, you should know that bigger does not automatically mean better or more popular.

For many people, their favourite search engine is simply the one they are introduced to first.  It's hard to say which search service is the most liked because of partnerships and some differences in counting visitors. 

There are different types of search services including search engines, directories, meta search engines and portals.   Each works in a different way.  Once you become familiar with the differences, you will find yourself changing from one type of service to another, depending on the sort of information you're looking for. 

In the rest of this piece, we will investigate how each type of service works and how to exploit these differences.

Search Engines

Search engines are software programmes that regularly collect information about Internet resources. A programme called a robot or spider or crawler is sent out to check the pages of sites it finds. With the help of another programme it then creates a catalogue or database describing all the pages it finds, listing each page name with various appropriate keywords.   It moves from link to link, cataloguing what it finds.

To access search engine software, you have to enter appropriate descriptive keywords. The search engine compares these keywords with those in its catalogue and then sends you a list of suitable matching links.  Search one for the word business and it will return every page it has found that contains the word business.

If you try to organise books on a shelf, you will soon realise that indexing and describing content is incredibly difficult.  Should you go by author's name, title, type of content, size of book, colour of cover…?   The range of choice is so vast, that all agents select and organise material according to different rules.  Some go first by title, others by word frequency, etc.

This in turn means that each search engine has its own quirky little ways.  The successful Internet researcher is aware of this and uses it to tweak searches for maximum efficiency.   To learn about each search engine's idiosyncrasies, look for the help notes offered when you submit your keywords.  Reading the help notes should take no longer than a few minutes and is a worthwhile investment.  It will help you be more efficient in your searches, eventually saving you a lot of time.

Note that search engines do not always catalogue everything.  Some specialise in particular types of resources.  One specialised search engine is Animal Crawler at http://www.animalcrawler.com that only indexes animal pages. 

Directories

A directory is a subject guide, typically organised by human editors into major topics and subtopics. Sites are listed under such categories as Art, Science, Shopping etc. Like printed directories, some online versions solicit payment for a web site being listed. Most, however, invite visitors to submit suitable sites for free.

Each directory has its own idiosyncrasies.  To learn about these, look for a link to the help notes offered on the welcome page.  Specialist directories are also quite common.  The Awesome Library at http://www.awesomelibrary.org for example, only lists child-safe educational resources. An example of a general directory is Open Directory at http://dmoz.org.

Many directories also offer on-site search facilities that sort through the links in the directory, but unlike search engines, directories do not send out robots to find and index new pages.  Instead, all links are sorted by human beings.  A well known directory with a search facility is About.Com at http://www.about.com.

Portals

Portals started as web sites that offered access to search engines. These web sites then evolved, offering extra features, such as free-email and directories, in order to attract advertising. Today's portals usually offer a directory, a search engine, a selection of current news items, weather information, free e-mail with the portal name incorporated, stock quotes, phone and map information, as well as community chat rooms and message boards.As a general rule of thumb portals tend to look like Yahoo! at http://www.yahoo.com. 

There are general portals and specialised or niche portals.  Examples of niche portals are Women.Com at http://www.women.com that targets female surfers and Malaysia Central at http://www.mycen.com.my that focuses on information of interest to Malaysian surfers.

Meta Search Sites

Known also as meta sites, meta crawlers, meta searchers, multisearch engine or "all in one search" pages, these services allow you to search various search engines and directories at the same time. A well known meta site is MetaCrawler at http://www. MetaCrawler.com

Which Search Service?

Search engines are useful for finding a specific resource, such as Jonathan Swift's A Modest Proposal.  Generally you just type in the name of the resource and the search engine will tell you where to find it online.  Search engines are also useful for finding resources that contain several specific keywords. If you're looking for documents that discuss electronic commerce in Spain for example, a search engine is the tool to use. 

If you have some keywords or a descriptive phrase and you are not sure which search engine to use, visit a meta search site.  For example, searching Dogpile at http://www.dogpile.com for the word lepak will return the results for this search at Google, Yahoo, Big and Ask all at the same time.  You can then choose the option that you think will be most fruitful.

Directories are good for surfing to see how much information is available on a particular topic.  Suppose you are given a choice of two essays. You can write about badminton or Sepak Takraw.   Can you expect to find an equal amount of information on each sport?  To find out, visit a directory like Open Directory at http://www.dmoz.org.  Click on Sports and surf your way through to the badminton section.  Check to see how many resources are available. Compare this to the information available on the sport Sepak Takraw.

Note that counting resources just shows quantity of information, not quality! Where subjects seem to have equal coverage, you will have to investigate further before determining which is the better option.

Directories are also a good way to find out what topics are the focus of attention within a certain discipline.  Say you are interested in architecture but don't know anything about it.  To find out what's hot, visit a directory, click on Architecture and then browse through the subcategories to find out more about the different schools, the leading lights past and present, which buildings are the most written about, what engineering topics are hotly debated and much more.  Surfing a directory in this way gives you a bird's eye view of a subject.

It is a good idea to perform several searches with different tools, just to see what is available.

Specialisation

Some search facilities specialise in particular types of resources, for example, HistorySeek! at http://www.buscopio.net/eng only indexes pages of interest to historians, genealogists and so on.

When you're searching the Net for a specific set of resources, it's useful to check for a particular search service or directory that is specialised in the subject you are interested in.There are two ways to do this: word of mouth, and by looking up what's listed by large general interest search facilities. As a shortcut, you use a search engine to search for the phrase "search engine" or "web directory" and the subject you are interested in.

A Note About Web Rings

Strictly speaking, web rings are not search facilities.  Also, they are no longer very popular.  However, those that remain are useful tools that can help you find information quickly and easily and therefore they are included them here.

A web ring is a way of interlinking related Web sites.  The idea is that when you stumble on something interesting, for example, a web site devoted to Pyramid Power and want to visit sites dedicated to the same burning issue, you don't have to search for similar resources.Look for the Web Ring icon.  From here you can explore one web site in the ring after another, or view the entire list at once and make your choice from there.

There are Web rings devoted to feng shui, acrobatics, electronic commerce, sewing, fishing, Jazz, and many more.  Some have just a few members, others have hundreds. Anyone may start a web ring.  For popular subjects there are more than one web ring. Each web ring has its own criteria for joining but there is no limit to the amount of web rings a site may be linked to.   A web site devoted to Hindu gods, for example, may be part of a religious web ring, a Hindu web ring and a web ring devoted to Asian myth all at the same time.  In practice, most sites belong to just one web ring.

For more information about web rings try http://www.webring.com and http://www.webringworld.org.

So what do I want to use?

In the final analysis, the difference between search services can be very fine. You may use one portal just for its directory and another portal for its search engine.  It doesn't matter which service or combination of services you use, as long as you can find what you want quickly and easily.

This page comes from my book Instant Genius: Internet Research. CIf you are curious to find out more about my work, check out my Writing Credits

My series Instant Genius! Information Technology Without The Technical Jargon that is published by Pelanduk is unfortunately out of print. I am considering a compilation and new publication.If you find this useful, please don't copy it and put it on your web site. Instead, please put in the first paragraph and a link to this page. I need the traffic! If you want a copy for a school intranet where kids can't go online in a general way, I don't mind, AS LONG AS... you keep the whole page intact, including the images of my book, my name and also give me a credit and link to lepak.com. And drop me an email to let me know? Thanks!

Also, please look at two of my books that are now out at bookshops in Malaysia, Singapore, Indonesia, Brunei and other countries.

Click on the images above for free sneak peek at chapters, prefaces, tables of contents, lists of retailers and other information.

All Rights Reserved 1998 - 2010