The Hearthseeker crawler

If you found this page from a line in your access log, this is what was asking and why. It has one name, it says so on every request, and the line that stops it is at the bottom.

The name it sends

That string never varies. The product token — Hearthseeker — is what a robots.txt line matches on, and it is fixed. The version number after the slash may change; your rule will keep working when it does.

Who runs it

Hearthseeker LLC, a one-person company in Mount Ephraim, New Jersey. We compile research reports about individual parcels of land from public records, for the person deciding whether to buy that parcel. Every finding in a report names the office that published it and the date it was read, so a buyer can go and check us.

What it reads

Published government land records: county and state map services, tax rolls, federal flood and soil surveys, census products, and open-data catalogs. It reads one parcel at a time, because somebody asked about that parcel.

It does not fill in forms, log in to anything, or post anywhere. It collects no personal information and runs no searches on people's names. It takes no copies of licensed text: where a record's own terms reserve it, we name the record and link to it rather than reproduce it.

How it behaves

Robots

It reads your robots.txt and honors it, including a rule aimed at every crawler rather than at us by name.

Rate

One request at a time to any one host, never in parallel, with a pause between them and a longer pause after an error. It does not run unattended loops against a county server.

Caching

Aggressively, because reading the same record twice is our cost as well as yours.

Hours

Whenever somebody asks about a parcel. There is no scheduled sweep of anybody's site.

How to stop it

Put this in your robots.txt and it will stop, on the next request, everywhere:

User-agent: Hearthseeker
Disallow: /

To allow part of a site and not the rest, the ordinary rules apply:

User-agent: Hearthseeker
Disallow: /
Allow: /gis/

If you would rather write to a person than edit a file, the address below reaches one, and a request to stop is honored without an argument about whether we were entitled to be there.

If something went wrong

If this crawler cost you traffic, tripped an alert, or asked for something it should not have, please say so. A rate that is fine for one server is too fast for another and we would rather be told than blocked. Tell us the host and roughly when, and we will find it in our own record of what we asked.

Contact

[email protected] — read by a person, not a queue.