eCairn Grabber is the web robot operated by eCairn.

It grabs and scans URLs (mainly blogs and other social properties like forums or Twitter accounts) selected by our users to bring them useful (and disclosed) information like the title of the site, the feeds that it may expose, the badges (Facebook, LinkedIn, …) and thus the social networks that the blogger may belong to. 

Once the feeds exposed by a site are successfully identified, the grabber periodically fetches them (one at a time) to allow our users to read the posts (RSS reader-like functionality).

How does the eCairn Grabber identify itself?

Our bot identifies itself with the following User-Agent string: “Mozilla/5.0 eCairn-Grabber/1.0 (+http://ecairn.com/grabber)”

Does the eCairn Grabber crawl my website?

No. Our bot does not crawl sites but only cares about the homepage and the feeds.

How often does the bot access pages?

On an average basis, our bot does not access the same HTML page more than once a day and the same XML feed more than 10 times a day, this frequency being reached for sites that publish posts at this or a higher rate.

From what IP addresses does your bot run?

We are currently grabbing pages and feeds from the following IP addresses:

• 54.209.100.60

• 3.95.242.114

• 18.234.30.142

• 174.129.3.130

• 107.23.221.122

• 52.90.151.137

• 54.158.125.130

Does the bot support conditional get/compression?

Yes. The eCairn Grabber supports conditional gets based on both Last-Modified and

eTag HTTP headers. It is built on top of cURL and, consequently, also supports compression.

Does the bot support the Robots Exclusion Standard/robots.txt protocol?

Not currently but it’s planned for a future release.

We do, however, maintain a “blacklist” of url that have told us not to scan them.

Whom can I contact with a question or concern about your grabber?

Please send all questions and feedback about eCairn Grabber to conversation+grabber(at)ecairn.com.