Crawler information
CivicProof identifies its crawler when checking public civic sources.
CivicProof is an independent civic records reader. Its crawler helps find public local government material such as agendas, minutes, packets, meeting notices, RSS feeds, and calendar feeds.
Identity
Crawler user agent
Site administrators may see this user agent in access logs:
CivicProofCrawler/1.0 (+https://www.civicproof.app/crawler; contact: hello@civicproof.app)
Behavior
How resident city checks access public sites
Resident discovery checks official records for one selected city within a limited request budget.
Uses sitemap hints
Discovery may read robots.txt to find sitemap URLs. The active city-records path does not interpret robots.txt Disallow or Crawl-delay directives.
Looks for public civic material
Discovery focuses on public agenda, minutes, packet, meeting notice, RSS, XML, and calendar feed links from official local government websites.
Uses bounded requests
City checks limit fetch time, redirects, candidate pages, and response sizes. Redirects are limited to the selected city’s allowed official and meeting-provider sites.
Stops on access barriers
The discovery path does not bypass logins, CAPTCHA, or other access controls. Access failures leave the source check incomplete or unavailable.
Contact
Questions or access concerns
If you administer a public website and need to report an issue, request a slower crawl rate, or ask CivicProof to avoid a specific public path, email hello@civicproof.app.