Call us on +44 (0)20 7100 6010 - click here to register

Instructing the Robots

It is useful to be able to give directions or hints to search engines as there are often parts of your website that do not need to be indexed, for example javascript files. The two main ways to give instructions are:

  • the Robots.txt file
  • the Robot meta tag

Robots.txt

This is a plain text file that tells search engine what not to bother with on your site. For example a folder that contains javascript, or one that you use to hold some development work in which isn't ready for public consumption. You place the file (called robots.txt) in your websites home directory.a typical file will look like this: 

User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /tmp/
Disallow: /private/

Robots Meta Tag

Use the Robot Meta tag to instruct the search engine what to do with a page. You can instruct them not to index the page and not to follow the links on a page.




Why not get in touch?

Products & Services

We develop marketing databases, campaign management tracking and analysis systems all the way to fully integrated web enabled CRM and ecommerce systems.

Performing Arts

If you’re finding marketing a challenge with so much information to manage – and in so many different places – you’ll find the Masque systems deliver exactly what you want..

Membership Organisations

From managing your day to day tasks to fully structured promotional projects, SMX enables you to achieve the most from your customer/supporter base.

Charity

Manage your relationship with your supporters as well as take donations both online and offline. SMX Fundraiser has advanced Gift Aid support.
Controlled Ordering Solutions & Rep Networks

From POS ordering to full sales process management - SMX will help you develop your customers