A continuation of Janady13/crawler-signal-reference / framework-html-meta-generator.md. The complete source identity, every recorded occurrence, and linked equations are on its first page.
Source transcript · continued
Download complete source text ↓1,893 lines · 60.0 KB
# Apply any fixes
find /var/www/sites/example.com/ -name "*.html" -type f -exec sed -i '/meta name="generator" content="WordPress/d' {} \;
find /var/www/sites/example.com/ -name "*.html" -type f -exec sed -i '/meta property="generator" content="Yoast/d' {} \;
# Reload nginx if configuration changed
nginx -t && systemctl reload nginx
```
### 17.5 The Client Migration Checklist
For every WordPress (or other CMS) migration:
1. [ ] Pre migration: document existing generator tags.
2. [ ] Build phase: new templates use only Bubbles brand statement.
3. [ ] Pre deploy: run `predeploy-generator-check.sh`.
4. [ ] Post deploy: run `disclosure-audit.sh`.
5. [ ] Verify HTTP headers: no X-Powered-By, no Server version.
6. [ ] Verify HTML: no CMS generator, no plugin signatures.
---
## 18. AUDIT CHECKLIST
Run through these 40 items for any production deployment.
### Core meta generator
1. [ ] No `<meta name="generator">` with version number on any indexable URL.
2. [ ] No `<meta name="generator">` with "WordPress", "Drupal", "Joomla", "Wix", "Squarespace" as content.
3. [ ] If generator tag present, it is a brand statement only (e.g., "Hand coded by ThatDeveloperGuy.com").
4. [ ] No `<meta property="generator">` (Yoast specific) on any page.
### Migration cleanup
5. [ ] Migrated sites have all old CMS generator tags removed.
6. [ ] No `wp-content` paths in HTML source.
7. [ ] No CMS specific signatures (plugin paths, theme paths).
8. [ ] No HTML comments revealing CMS versions.
### HTTP layer
9. [ ] `server_tokens off;` in nginx.conf.
10. [ ] `Server: nginx` (no version) in response headers.
11. [ ] No `X-Powered-By` header in responses.
12. [ ] No `X-AspNet-Version` header.
13. [ ] No `X-Runtime` header.
### FastAPI specific
14. [ ] CleanupHeadersMiddleware applied (removes leaked headers).
15. [ ] No version disclosure in FastAPI default behavior.
### Site type specific
16. [ ] Hand coded Bubbles sites: brand statement OR omit.
17. [ ] Federal subcontractor sites: omit entirely.
18. [ ] Anonymous client sites: omit entirely.
19. [ ] Client preference sites: per client preference.
### CMS configuration (where still managed)
20. [ ] WordPress sites: functions.php hook removes generator.
21. [ ] Hugo sites: `disableHugoGeneratorInject = true`.
22. [ ] Jekyll sites: layout template doesn't include generator.
23. [ ] Gatsby sites: gatsby-config.js suppresses generator.
### Verification commands tested
24. [ ] `curl -s URL | grep generator` returns expected (brand statement or empty).
25. [ ] `curl -sI URL | grep -i "x-powered"` returns nothing.
26. [ ] `curl -sI URL | grep -i "^server"` returns "server: nginx" only.
### Bulk audit
27. [ ] `generator-audit.sh` shows no version disclosures across all client sites.
28. [ ] `disclosure-audit.sh` clean for each client site.
### Pre deploy gates
29. [ ] CI/CD or manual pre deploy check verifies no version in generator.
30. [ ] Template review rejects new version disclosures.
### Documentation
31. [ ] Per client generator decision documented.
32. [ ] Migration cleanup process documented.
33. [ ] WordPress hardening hooks documented.
### Cross cutting
34. [ ] No version disclosure in robots.txt comments.
35. [ ] No version disclosure in sitemap comments.
36. [ ] No version disclosure in HTML comments.
37. [ ] No version disclosure in JavaScript console output.
38. [ ] No version disclosure in CSS comments.
### Quarterly review
39. [ ] Quarterly audit run across all client sites.
40. [ ] New CVEs reviewed; sites with version disclosure prioritized for cleanup if any remain.
A site that passes all 40 has correctly handled generator disclosure: no version numbers, no CMS revelations, only brand statements where appropriate, with HTTP layer hardening in place.
---
## 19. COMMON PITFALLS
Twelve patterns to recognize and avoid.
**Pitfall 1: Migrated WordPress site still has WordPress generator.**
Symptom: page source shows `<meta name="generator" content="WordPress 6.4.2">` despite being hand coded.
Why it breaks: migration didn't strip the old tag.
Fix: remove via sed across all HTML files; verify with grep.
**Pitfall 2: Generator tag with version number.**
Symptom: `<meta name="generator" content="WordPress 6.4.2">` or `<meta name="generator" content="Hugo 0.140.0">`.
Why it breaks: enables CVE targeting.
Fix: remove version; either omit entirely or use brand statement without version.
**Pitfall 3: nginx server_tokens not set.**
Symptom: `Server: nginx/1.26.0` in response headers.
Why it breaks: nginx version disclosed; vulnerable to CVE targeting.
Fix: `server_tokens off;` in nginx.conf.
**Pitfall 4: X-Powered-By header present.**
Symptom: `X-Powered-By: PHP/8.2.4` or `X-Powered-By: Express` in responses.
Why it breaks: backend framework and version disclosed.
Fix: remove via FastAPI middleware or nginx `proxy_hide_header X-Powered-By`.
**Pitfall 5: HTML comments revealing plugin versions.**
Symptom: `<!-- Yoast SEO v21.7 -->` in HTML.
Why it breaks: plugin version disclosure; specific vulnerability targeting.
Fix: remove via WordPress functions.php hooks or post processing.
**Pitfall 6: Yoast `<meta property="generator">` not removed.**
Symptom: `<meta property="generator" content="Yoast SEO v21.7">` (uses property not name).
Why it breaks: separate from name="generator"; needs separate removal.
Fix: `add_filter('wpseo_debug_marker', '__return_empty_string');`
**Pitfall 7: Generator tag in noindex pages.**
Symptom: login pages, admin pages have generator tags.
Why it breaks: unnecessary disclosure; still scrapable.
Fix: omit from all pages, even noindex.
**Pitfall 8: Generator as keywords stuffing alternative.**
Symptom: `<meta name="generator" content="WordPress, Yoast, SEO optimized site">`.
Why it breaks: misuse of the field; doesn't help with SEO.
Fix: use generator only for actual tool identification or brand statement.
**Pitfall 9: Federal site has generator with CMS identification.**
Symptom: federal subcontractor site discloses Drupal version.
Why it breaks: Section 508/NIST 800-53 audit finding.
Fix: omit generator entirely for federal sites.
**Pitfall 10: Generator differs across pages on same site.**
Symptom: homepage has generator A, blog has generator B.
Why it breaks: inconsistent signal; suggests template fragmentation.
Fix: consistent generator (or none) across all templates.
**Pitfall 11: WordPress plugins overriding the wp_generator removal.**
Symptom: `wp_generator` removed but generator tag still appears.
Why it breaks: another plugin or theme is re adding it.
Fix: track down the source; remove with higher priority filter or block specifically.
**Pitfall 12: Generator brand statement contains contact info.**
Symptom: `<meta name="generator" content="Joseph Anady joseph@thatdeveloperguy.com 505-512-3662">`.
Why it breaks: privacy concern; spam harvesting; clutter.
Fix: brand name only. Contact info goes in body content or schema.
---
## 20. DIAGNOSTIC COMMANDS
Reference of commands useful for generator investigation.
### Inspect a single URL
```bash
# Extract meta generator
curl -s https://example.com/ | grep -oE 'meta name="generator"[^>]+' | head -1
# Extract Yoast property generator
curl -s https://example.com/ | grep -oE 'meta property="generator"[^>]+' | head -1
# Check for version in generator
curl -s https://example.com/ | grep -oE 'meta name="generator"[^>]+' | grep -E '[0-9]+\.[0-9]+'
# Check for CMS name in generator
curl -s https://example.com/ | grep -ioE 'meta name="generator"[^>]+' | grep -iE 'wordpress|drupal|joomla|wix|squarespace'
```
### Inspect HTTP layer
```bash
# Check Server header
curl -sI https://example.com/ | grep -i "^server"
# Check X-Powered-By
curl -sI https://example.com/ | grep -i "x-powered-by"
# Comprehensive disclosure audit
curl -sI https://example.com/ | grep -iE "server:|x-powered|x-aspnet|x-runtime"
```
### Bulk audit a sitemap
```bash
curl -s https://example.com/sitemap.xml | grep -oE "<loc>[^<]+</loc>" | sed 's/<[^>]*>//g' | while read url; do
GENERATOR=$(curl -s "$url" 2>/dev/null | grep -oE 'meta name="generator"[^>]+' | head -1)
if [ -n "$GENERATOR" ]; then
if echo "$GENERATOR" | grep -qE '[0-9]+\.[0-9]+'; then
echo "VERSION DISCLOSED: $url"
echo " $GENERATOR"
fi
fi
done
```
### Cross site audit
```bash
generator-audit.sh
```
### Detailed disclosure audit per URL
```bash
disclosure-audit.sh https://example.com/
```
### After cleanup
```bash
# Apply changes (HTML)
nginx -t && systemctl reload nginx
# Or restart FastAPI
systemctl restart fastapi-sidecar
# Verify
disclosure-audit.sh https://example.com/
```
### WordPress specific
```bash
# Check WordPress site for all version disclosures
SITE=https://example-wp.com
echo "=== Generator ==="
curl -s "$SITE/" | grep -oE 'meta name="generator"[^>]+\|meta property="generator"[^>]+'
echo "=== HTML comments ==="
curl -s "$SITE/" | grep -oE '<!--[^>]*[0-9]+\.[0-9]+[^>]*-->' | head -5
echo "=== Plugin signatures ==="
curl -s "$SITE/" | grep -oE 'wp-content/plugins/[^/]+' | sort -u | head -10
echo "=== HTTP headers ==="
curl -sI "$SITE/" | grep -iE "server|x-powered"
```
### Browser based check
In Chrome DevTools:
1. Open the page.
2. View Source (Cmd/Ctrl + U).
3. Search (Cmd/Ctrl + F) for "generator".
4. Search for version patterns: regex `\d+\.\d+`.
5. Network tab: inspect Response Headers for Server, X-Powered-By.
Use Wappalyzer browser extension:
- Install Wappalyzer.
- Visit the site.
- See what tech is detected.
- For Bubbles sites: should show "Custom" or no platform.
---
## 21. CROSS-REFERENCES
* [framework-http-performance-headers.md](framework-http-performance-headers.md) Section 4 (Server header): the HTTP layer equivalent; `server_tokens off;` directive for nginx.
* [framework-http-security-headers.md](framework-http-security-headers.md): X-Powered-By removal as part of broader security hardening.
* [framework-html-meta-author.md](framework-html-meta-author.md): the brand statement in generator complements author attribution; together they signal authorship.
* [framework-html-meta-keywords.md](framework-html-meta-keywords.md): like meta keywords, generator is exposed in page source; competitive intelligence implications.
* [framework-html-meta-charset.md](framework-html-meta-charset.md): generator tag placement after charset; within first 1024 bytes irrelevant for generator but consistent ordering matters.
* [UNIVERSAL-RANKING-FRAMEWORK.md](UNIVERSAL-RANKING-FRAMEWORK.md): the master ranking reference. Generator ignored by Google means alternative ranking signals do the work.
* [SEO-BUILD-REFERENCE.md](SEO-BUILD-REFERENCE.md) v2.4: the build playbook including security hardening.
* MDN meta name=generator: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta/name
* NIST National Vulnerability Database: https://nvd.nist.gov
* CIS Benchmarks (information disclosure baselines): https://www.cisecurity.org/cis-benchmarks
* OWASP Information Disclosure: https://owasp.org/www-community/Improper_Error_Handling
* WordPress Codex on the_generator filter: https://developer.wordpress.org/reference/hooks/the_generator/
* Hugo configuration reference: https://gohugo.io/getting-started/configuration/
* nginx server_tokens directive: https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens
* Wappalyzer: https://www.wappalyzer.com
* BuiltWith: https://builtwith.com
---
## APPENDIX A: ONE PAGE QUICK REFERENCE
For the person who just wants the answer.
### The Bubbles rule
**Never include version numbers in generator. Default: brand statement OR omit.**
### The canonical patterns
```html
<!-- Default (hand coded Bubbles site): -->
<meta name="generator" content="Hand coded by ThatDeveloperGuy.com">
<!-- For federal/anonymous clients: omit entirely -->
<!-- (no meta generator tag) -->
```
### The forbidden patterns
```html
<!-- NEVER USE (version disclosure) -->
<meta name="generator" content="WordPress 6.4.2">
<meta name="generator" content="Hugo 0.140.0">
<meta name="generator" content="Drupal 10.1.5">
```
### The HTTP layer companion
```nginx
# Required for every Bubbles nginx installation
server_tokens off;
```
```python
# Required for every Bubbles FastAPI sidecar
class HardenHeadersMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request, call_next):
response = await call_next(request)
response.headers.pop("X-Powered-By", None)
return response
```
### Five rules to memorize
1. **Never disclose versions in generator (CVE exploitation invitation).**
2. **WordPress migrations require explicit cleanup of WordPress generator.**
3. **Brand statement "Hand coded by ThatDeveloperGuy.com" is optional but useful.**
4. **HTTP layer hardening (server_tokens off, X-Powered-By removed) is mandatory.**
5. **Federal subcontracting work: omit generator entirely (security baseline).**
### Five commands every operator should know
```bash
# 1. Check meta generator
curl -s URL | grep -oE 'meta name="generator"[^>]+'
# 2. Check HTTP disclosures
curl -sI URL | grep -iE "server|x-powered"
# 3. Bulk audit Bubbles client sites
generator-audit.sh
# 4. Remove WordPress generator from all HTML files
find /var/www/sites/example.com/ -name "*.html" -exec sed -i '/meta name="generator" content="WordPress/d' {} \;
# 5. Apply nginx changes
nginx -t && systemctl reload nginx
```
### Three end to end tests
```bash
# 1. No version disclosed in generator
GEN=$(curl -s https://example.com/ | grep -oE 'meta name="generator"[^>]+' | head -1)
echo "$GEN" | grep -qE '[0-9]+\.[0-9]+' && echo "FAIL: version disclosed" || echo "OK"
# 2. No CMS disclosed in generator
echo "$GEN" | grep -qiE 'WordPress|Drupal|Joomla|Wix|Squarespace' && echo "FAIL: CMS disclosed" || echo "OK"
# 3. HTTP hardened
HTTP_DISCL=$(curl -sI https://example.com/ | grep -iE "x-powered|server: nginx/")
[ -z "$HTTP_DISCL" ] && echo "OK: HTTP hardened" || echo "FAIL: $HTTP_DISCL"
```
If all three pass AND `disclosure-audit.sh` reports clean AND nginx logs show no targeted CVE scan attempts, the generator layer is correctly handled.
---
End of framework-html-meta-generator.md.