Streamline3Bot

What is Streamline3Bot?

Streamline3Bot is a web crawler maintained by UBT.

UBT provides a classification service for public websites based on their content.  To classify your website, we use the Streamline3Bot to crawl the web on a daily basis.

To classify web sites Streamline3bot checks each site approximately once every three days - the frequency at which we crawl your site may vary according to the frequency at which your site is updated, and the classification that the crawler attributes to your site.

When the Streamline3Bot crawls your site, it can be identified by Streamline3Bot in the User-Agent request header.

The user agent is:

Verifying Streamline3Bot

You can verify that the Streamline3Bot is accessing your site - this maybe useful if you consider that another crawler is masquerading as the Streamline3Bot.

UBT does not publish a public list of IP addresses for webmasters to whitelist. This is because these IP address ranges change regularly.  For this reason, we strongly recommend that webmasters do not hard-code, or specifically whitelist IP addresses pertaining to the Streamline3Bot.

To verify the Streamline3Bot, you must first run a DNS lookup as described below.

To verify Streamline3Bot as the crawler:

1. Identify the IP address that is performing the site crawl – you will find this in the server access logs.
2. Run a reverse DNS lookup on the IP address using the 'host' command.
3. Verify that the domain name is in the ubtsupport.com domain.
4. Run a forward DNS lookup on the domain name retrieved at step 2 using the ‘host’ command on the retrieved domain name.
5. Verify that it is the same as the IP address identified at step 1.

Example:

Questions

If you have any questions relating to the Streamline3Bot, please don’t hesitate to write to us at:

UBT (EU) Ltd
Exchange Place
Poseidon Way
Warwick
Warwickshire
CV34 6BY