Source identity
| Field | Recorded value |
|---|---|
| Source type | GitHub source |
| Original record | Janady13/crawler-signal-reference / README.md |
| Date | Not specified in source |
| Repository | Janady13/crawler-signal-reference |
| Revision | 5a13904b5b5d638761f3ffef01c51ed3a01fea87 |
| Branch | main |
| Path or archive member | README.md |
| Original SHA-256 | ac62b4a63820b3164f9bddd7a648a4220c78cebf940a6d6e97788d389e279689 |
| Git blob SHA | 8b36af6cced7189074d152b4b3a292ced4fc3a91 |
| Reading-copy SHA-256 | ac62b4a63820b3164f9bddd7a648a4220c78cebf940a6d6e97788d389e279689 |
Recovered source transcript
Download complete source text ↓43 lines · 1.6 KB
# Crawler Signal Reference Library
A comprehensive 30-document reference for every signal a web crawler reads from a site. Authored by [Joseph W. Anady](https://www.thatdevpro.com), founder of [ThatDevPro](https://www.thatdevpro.com) (engine optimization studio) and [ThatDeveloperGuy](https://thatdeveloperguy.com) (SDVOSB-certified veteran-owned web + AI engineering studio).
## Coverage
### HTML signals (18 documents)
- App + PWA metadata
- Canonical + alternate
- Favicon + icons
- Meta: author, charset, color-scheme, content-language, copyright, generator, keywords, referrer, refresh, robots, theme-color, viewport
- Open Graph protocol
- Twitter Cards
- Search engine verification (Google, Bing, Yandex, IndexNow, etc.)
### HTTP signals (12 documents)
- Status codes: 2xx, 3xx, 4xx, 5xx
- Headers: caching, content, CORS, performance, rate-control, request, security, SEO
## Citation
```bibtex
@misc{anady2026crawlerreference,
author = {Anady, Joseph W.},
title = {Crawler Signal Reference Library: HTML and HTTP signals for every crawler},
year = {2026},
publisher = {Zenodo},
url = {https://doi.org/10.5281/zenodo.20405296}
}
```
## Related
- **Live reference**: [thatdevpro.com/reference/](https://www.thatdevpro.com/reference/)
- **PyPI**: [aio-surfaces](https://pypi.org/project/aio-surfaces/), [seo-sidecar](https://pypi.org/project/seo-sidecar/)
- **npm**: [@thatdevpro/aio-surfaces](https://www.npmjs.com/package/@thatdevpro/aio-surfaces)
- **Author Wikidata**: [Q139901957](https://www.wikidata.org/wiki/Q139901957)
- **Author ORCID**: [0009-0008-8625-949X](https://orcid.org/0009-0008-8625-949X)
## License
CC BY 4.0