CivicProof

Crawler information

CivicProof identifies its crawler when checking public civic sources.

CivicProof is an independent civic records reader. Its crawler helps find public local government material such as agendas, minutes, packets, meeting notices, RSS feeds, and calendar feeds.

Identity

Crawler user agent

Site administrators may see this user agent in access logs:

CivicProofCrawler/1.0 (+https://www.civicproof.app/crawler; contact: hello@civicproof.app)

Behavior

How CivicProof crawls public sites

Discovery is designed to be conservative, bounded, and respectful of municipal website limits.

  1. Checks robots rules first

    The crawler reads robots.txt before probing public pages, honors disallowed paths, and reports crawl-delay directives so CivicProof can account for them before production use.

  2. Looks for public civic material

    Discovery focuses on public agenda, minutes, packet, meeting notice, RSS, XML, and calendar feed links from official local government websites.

  3. Uses bounded requests

    Requests use timeouts, same-origin redirect checks, small batches, and per-domain pacing so public websites are not hit aggressively.

  4. Stops on access barriers

    CivicProof does not bypass logins, CAPTCHA, access controls, or human-only defenses. Blocked or protected pages are routed for manual review.

Contact

Questions or access concerns

If you administer a public website and need to report an issue, request a slower crawl rate, or ask CivicProof to avoid a specific public path, email hello@civicproof.app.