A continuation of Janady13/crawler-signal-reference / framework-http-3xx-status-codes.md. The complete source identity, every recorded occurrence, and linked equations are on its first page.
Source transcript · continued
Download complete source text ↓2,209 lines · 79.3 KB
async def login(form: LoginForm, return_to: str = "/"):
if await authenticate(form):
safe_target = safe_redirect_target(return_to)
return RedirectResponse(url=safe_target, status_code=303)
raise HTTPException(status_code=401)
```
### 18.4 Maintenance window (302 with Retry-After)
```nginx
server {
location / {
return 302 /maintenance;
}
location = /maintenance {
root /var/www/sites/example.com;
try_files /maintenance.html =503;
add_header Retry-After "3600" always;
add_header Cache-Control "no-store" always;
}
}
```
After maintenance: remove the `return 302 /maintenance;` and reload nginx.
### 18.5 Post/redirect/get pattern (303)
```python
@app.post("/checkout")
async def checkout(order: OrderForm):
order_id = await process_order(order)
return RedirectResponse(
url=f"/orders/{order_id}/confirmation",
status_code=303
)
@app.get("/orders/{order_id}/confirmation")
async def confirmation(order_id: str):
order = await get_order(order_id)
return render_confirmation(order)
```
### 18.6 API endpoint permanent migration with method preservation (308)
```python
@app.api_route("/api/v1/users", methods=["GET", "POST", "PUT", "DELETE"])
async def v1_users_migrated():
return RedirectResponse(
url="/api/v2/users",
status_code=308
)
```
### 18.7 API endpoint temporary migration with method preservation (307)
```python
@app.api_route("/api/v1/payments", methods=["POST", "PUT"])
async def v1_payments_temp():
return RedirectResponse(
url="/api/v1/payments-backup",
status_code=307
)
```
### 18.8 Conditional GET for dynamic content (304)
```python
from email.utils import formatdate
import hashlib
@app.get("/api/data")
async def api_data(request: Request):
content = generate_data()
body = json.dumps(content).encode()
etag = '"' + hashlib.md5(body).hexdigest()[:16] + '"'
inm = request.headers.get("if-none-match", "")
if etag in [tag.strip() for tag in inm.split(",") if tag.strip()]:
return Response(
status_code=304,
headers={"ETag": etag, "Cache-Control": "public, max-age=60"}
)
return Response(
content=body,
media_type="application/json",
headers={"ETag": etag, "Cache-Control": "public, max-age=60"}
)
```
### 18.9 Sitemap migration: redirect old sitemap to new
```nginx
location = /sitemap.xml {
return 301 /sitemap-new.xml;
}
location = /sitemap-new.xml {
root /var/www/sites/example.com;
try_files $uri =404;
add_header Content-Type "application/xml" always;
}
```
### 18.10 Safe internal redirect helper
```python
from urllib.parse import urlparse
def safe_redirect_target(target: str, default: str = "/") -> str:
"""Return target if it is a safe internal path, else default."""
if not target:
return default
if not target.startswith("/"):
return default
if target.startswith("//"): # protocol relative URL
return default
if target.startswith("/\\"): # backslash trick
return default
parsed = urlparse(target)
if parsed.scheme or parsed.netloc:
return default
return target
```
### 18.11 Bubbles subdomain to canonical pattern (from Section 16)
See Section 16.2 for the complete reference block.
### 18.12 Trailing slash canonicalization
Pick one (with or without trailing slash) and redirect the other:
```nginx
# Option A: enforce no trailing slash
rewrite ^/(.*)/$ /$1 permanent;
# Option B: enforce trailing slash on directories only
location / {
try_files $uri $uri/ $uri.html =404;
}
```
### 18.13 Removed section permanent redirect to category
```nginx
# Old section removed; redirect to category
location /old-section/ {
return 301 /category/related;
}
```
Note: redirect to a relevant category, NOT to the homepage. Redirecting deleted pages to the homepage is the soft 404 trap (covered in framework-http-2xx-status-codes.md).
---
## 19. BUBBLES NGINX REFERENCE BLOCK (PASTE READY)
The complete 3xx aware configuration for a Bubbles client site.
```nginx
# ============= HTTP to HTTPS canonical (single 301) =============
server {
listen 80;
listen [::]:80;
server_name example.com www.example.com;
return 301 https://example.com$request_uri;
}
# ============= www HTTPS to non www HTTPS (separate single 301) =============
server {
listen 443 ssl;
listen [::]:443 ssl;
http2 on;
server_name www.example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
return 301 https://example.com$request_uri;
}
# ============= CANONICAL SERVER =============
server {
listen 443 ssl;
listen 443 quic;
listen [::]:443 ssl;
listen [::]:443 quic;
http2 on;
http3 on;
server_name example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_early_data on;
add_header Alt-Svc 'h3=":443"; ma=86400' always;
include snippets/common-security-headers.conf;
root /var/www/sites/example.com;
index index.html;
# ============= LEGACY URL 301 REDIRECTS =============
# Each legacy URL: single 301 to final destination
location = /old-home {
return 301 /;
}
location = /services-page {
return 301 /services/;
}
location ~* ^/blog/[0-9]+/[0-9]+/(.+)$ {
return 301 /articles/$1;
}
location ~* ^/old-product/(.+)$ {
return 301 /products/$1;
}
# ============= TRAILING SLASH NORMALIZATION =============
# Pick: with slash or without. Below: enforce with slash on directories
# No explicit redirect rule; nginx try_files handles via 404 if mismatched
# ============= API ENDPOINT MIGRATION (308 preserves method) =============
location = /api/v1/users {
return 308 /api/v2/users;
}
# ============= STATIC ASSETS (conditional GET produces 304 automatically) =============
location ~* \.(css|js|woff2|jpg|jpeg|png|webp|avif|svg)$ {
add_header Cache-Control "public, max-age=31536000, immutable" always;
# nginx returns 304 automatically on If-None-Match / If-Modified-Since match
}
# ============= HTML PAGES =============
location ~* \.html$ {
add_header Cache-Control "public, max-age=0, must-revalidate" always;
# nginx returns 304 if client validates with current ETag
}
# ============= LOGIN FLOW (302 with safe return_to) =============
location = /dashboard {
# Upstream handles 302 to /login?return_to=/dashboard
proxy_pass http://127.0.0.1:9090;
}
location = /login {
proxy_pass http://127.0.0.1:9090;
# FastAPI handles authentication and 303 post-redirect-get
}
# ============= API =============
location /api/ {
if ($request_method = OPTIONS) {
add_header Access-Control-Allow-Origin $cors_origin always;
return 204;
}
add_header X-Robots-Tag "noindex, nofollow" always;
proxy_pass http://127.0.0.1:9090;
}
# ============= ROOT =============
location / {
try_files $uri $uri/ $uri.html =404;
}
}
```
After deploying:
```bash
nginx -t && systemctl reload nginx
```
Verify the redirect chain is exactly one hop:
```bash
# HTTP to canonical HTTPS: should be exactly one redirect
curl -sIL http://example.com/ | grep -c "^HTTP"
# Expected: 2 (the 301 + the final 200)
# WWW HTTPS to non-WWW HTTPS: also exactly one redirect
curl -sIL https://www.example.com/ | grep -c "^HTTP"
# Expected: 2 (the 301 + the final 200)
# Legacy URL to new URL: also exactly one redirect
curl -sIL https://example.com/old-home | grep -c "^HTTP"
# Expected: 2 (the 301 + the final 200)
# Trace each redirect explicitly
curl -sIL https://example.com/old-home | grep -E "^HTTP|^Location:"
```
---
## 20. AUDIT CHECKLIST
Run through these 50 items for any production deployment.
### Canonical redirect setup
1. [ ] HTTP to HTTPS uses 301 (not 302).
2. [ ] HTTP to HTTPS is a single hop to the canonical hostname.
3. [ ] WWW to non WWW (or vice versa) uses 301.
4. [ ] WWW HTTPS to non WWW HTTPS is a single hop.
5. [ ] HSTS is set on the canonical HTTPS responses.
6. [ ] Trailing slash policy is consistent (chosen and enforced).
### Permanent redirects (301/308)
7. [ ] All site migrations use 301 (or 308 if method preservation needed).
8. [ ] API endpoint migrations use 308 (method preserved).
9. [ ] Legacy URLs redirect to relevant new URLs, NOT to the homepage.
10. [ ] No 301 redirects to error pages or thin content (would create soft 404 chains).
11. [ ] All 301 redirects verified working with curl -sIL.
12. [ ] Internal links updated to point to the final URL, not via redirect.
13. [ ] Sitemap lists final URLs, not redirected ones.
14. [ ] Canonical tags point to final URLs.
### Temporary redirects (302/307)
15. [ ] Login flows use 302 or 303.
16. [ ] Maintenance redirects use 302 with Retry-After.
17. [ ] No 302 redirects in place longer than 90 days without review.
18. [ ] A/B test redirects use 302 (not 301).
19. [ ] Geo redirects use 302 (canonical URL stays in index).
### Method preserving redirects (307/308)
20. [ ] API endpoints that accept POST/PUT/PATCH/DELETE use 307 (temp) or 308 (perm).
21. [ ] Form action endpoints that may be redirected use 307/308.
22. [ ] Webhook receiver endpoints use 308 for permanent moves.
### 303 (post/redirect/get)
23. [ ] Form submissions follow PRG pattern: POST to 303 to GET.
24. [ ] No double submission on browser refresh.
### 304 (conditional GET)
25. [ ] Static assets return 304 when If-None-Match matches.
26. [ ] Static assets return 304 when If-Modified-Since matches.
27. [ ] Dynamic responses implement conditional GET where appropriate.
28. [ ] 304 responses include current ETag (or Last-Modified).
29. [ ] 304 responses have empty body.
30. [ ] ETag generation is content based (not timestamp based).
### Chain prevention
31. [ ] No redirect chains exceeding one hop.
32. [ ] Audit script run periodically: `for url in $(sitemap); do count hops; done`.
33. [ ] Mixed redirect types (301to302) eliminated; one type per chain.
### Security
34. [ ] No open redirect on any endpoint accepting URL parameters.
35. [ ] If redirect endpoint exists: allow list OR path only OR signed.
36. [ ] `safe_redirect_target()` helper used in FastAPI sidecar.
37. [ ] No redirects to user supplied protocol relative URLs (//).
### Bubbles specific
38. [ ] The 14 paying client subdomain 301s verified working.
39. [ ] Wildcard subdomain noindex applied to non paying tier.
40. [ ] AdSense not injected on the 301 responses (only on serving demo content).
### Verification
41. [ ] `curl -sIL` confirms expected single hop on every redirect.
42. [ ] HSTS header present on HTTPS responses after the 301.
43. [ ] GSC Page Indexing shows no redirect chain errors.
44. [ ] Screaming Frog or similar shows no chains exceeding one hop.
45. [ ] Browser DevTools Network tab shows single hop on critical paths.
### Cross cutting
46. [ ] nginx -t passes without warnings.
47. [ ] nginx -T | grep "return 30" shows expected status codes.
48. [ ] No accidental 302 where 301 is intended.
49. [ ] Monitoring alerts on unexpected redirect status codes.
50. [ ] Quarterly review of all redirects to verify they are still needed.
A site that passes all 50 has correctly configured 3xx redirects for SEO equity, API correctness, security, and crawl efficiency.
---
## 21. COMMON PITFALLS
Fifteen patterns to recognize and avoid.
**Pitfall 1: 302 used for permanent migration.**
Symptom: new domain not ranking; old URL stays in search results.
Why it breaks: 302 is weak signal; Google keeps old URL canonical.
Fix: change to 301. Submit Change of Address in GSC.
**Pitfall 2: 301 used for temporary maintenance redirect.**
Symptom: site URL "stuck" at maintenance page even after maintenance ends.
Why it breaks: browsers and crawlers cache 301 aggressively.
Fix: use 302 (or 503 with Retry-After) for temporary redirects.
**Pitfall 3: Redirect chain (4+ hops).**
Symptom: slow page loads; crawl budget wasted.
Why it breaks: each hop is a round trip.
Fix: collapse chains. Every old URL to final URL in one hop.
**Pitfall 4: POST endpoint redirected with 301; data lost.**
Symptom: form submissions or API calls result in empty data at new endpoint.
Why it breaks: browsers convert POST to GET on 301.
Fix: use 308 instead.
**Pitfall 5: Open redirect on `/redirect?url=`.**
Symptom: phishing campaigns abuse the domain; deliverability collapses.
Why it breaks: any external URL accepted.
Fix: allow list, path only, or signed redirects.
**Pitfall 6: 301 to homepage for all removed pages.**
Symptom: removed product pages all redirect to homepage; soft 404 by Google.
Why it breaks: irrelevant destination; Google detects no canonical match.
Fix: 410 for permanently removed pages, or 301 to a relevant category.
**Pitfall 7: 304 returned with body.**
Symptom: some HTTP clients fail to parse subsequent responses.
Why it breaks: RFC says 304 MUST have no body.
Fix: return empty body from server.
**Pitfall 8: 304 missing ETag header.**
Symptom: client cannot determine which version 304 refers to.
Why it breaks: ETag is the validator; without it, next revalidation fails.
Fix: always include current ETag on 304 response.
**Pitfall 9: HTTP to HTTPS redirect is two hops (HTTP to HTTPS www to HTTPS canonical).**
Symptom: extra latency on every first visit.
Why it breaks: two separate 301s instead of one.
Fix: redirect HTTP directly to canonical HTTPS in one step.
**Pitfall 10: `return 302` with `if` containing `proxy_pass`.**
Symptom: redirect intermittent or wrong.
Why it breaks: nginx `if + proxy_pass` antipattern.
Fix: use map or location matching instead of conditional if blocks.
**Pitfall 11: 302 for HTTPS canonical (instead of 301).**
Symptom: HTTPS variant not picked up as canonical.
Why it breaks: 302 is weak signal; original (HTTP) stays canonical in Google's mind.
Fix: 301.
**Pitfall 12: Conditional GET ignored on dynamic responses.**
Symptom: crawlers download full body on every visit.
Why it breaks: upstream code does not check If-None-Match / If-Modified-Since.
Fix: implement conditional GET in upstream (Section 9.4).
**Pitfall 13: Browser cached 301 prevents rollback.**
Symptom: changing the server returns the new behavior, but browsers still follow the old 301.
Why it breaks: 301 cached client side persistently.
Fix: change the URL itself (cache miss on new URL). Or wait for cache expiry.
**Pitfall 14: Redirect destination is 404.**
Symptom: 301 to a page that no longer exists.
Why it breaks: chain ends at 404; bad UX, lost SEO equity.
Fix: audit redirect destinations periodically; remove or update broken ones.
**Pitfall 15: HSTS preloaded but redirect target is HTTP.**
Symptom: certificate error or refused connection on first visit.
Why it breaks: HSTS forces HTTPS but redirect points to HTTP.
Fix: all redirect targets must be HTTPS.
---
## 22. DIAGNOSTIC COMMANDS
Reference of every command useful for 3xx investigation.
### Inspect redirect behavior
```bash
# Show redirect chain
curl -sIL https://example.com/old-url | grep -E "^HTTP|^Location:"
# Count hops
HOPS=$(curl -sIL https://example.com/old-url | grep -c "^HTTP")
echo "Hops: $((HOPS - 1))" # subtract final 200
# Get just the redirect status and destination
curl -sI https://example.com/old-url | head -3
```
### Audit all redirects in a sitemap
```bash
# Find any URLs that redirect (should be none in a healthy sitemap)
SITEMAP=https://example.com/sitemap.xml
curl -s "$SITEMAP" | grep -oE "<loc>[^<]+</loc>" | sed 's/<[^>]*>//g' | \
while read url; do
STATUS=$(curl -so /dev/null -w "%{http_code}" "$url")
if [ "$STATUS" != "200" ]; then
DEST=$(curl -sI "$url" | grep -i "^location:" | awk '{print $2}' | tr -d '\r')
echo "$STATUS $url -> $DEST"
fi
done
```
### Find chains exceeding one hop
```bash
# Audit sitemap for chains
curl -s https://example.com/sitemap.xml | grep -oE "<loc>[^<]+</loc>" | sed 's/<[^>]*>//g' | \
while read url; do
HOPS=$(curl -sIL "$url" 2>/dev/null | grep -c "^HTTP")
if [ "$HOPS" -gt 2 ]; then
echo "CHAIN ($((HOPS - 1)) hops): $url"
fi
done
```
### Test method preservation
```bash
# 301: method changes (typical browser behavior)
curl -sIL -X POST -d "key=value" https://example.com/301-endpoint 2>&1 | \
grep -E "^> POST|^> GET|^HTTP|^Location"
# Expect: POST becomes GET on the followup
# 308: method preserved
curl -sIL -X POST -d "key=value" https://example.com/308-endpoint 2>&1 | \
grep -E "^> POST|^> GET|^HTTP|^Location"
# Expect: POST remains POST on the followup
```
### Test conditional GET (304)
```bash
# Get current ETag
ETAG=$(curl -sI https://example.com/style.css | grep -i "^etag:" | awk '{print $2}' | tr -d '\r')
# Verify 304 returned with matching ETag
curl -sI -H "If-None-Match: $ETAG" https://example.com/style.css | head -3
# Expected: HTTP/2 304
# Verify body is empty
BYTES=$(curl -s -H "If-None-Match: $ETAG" https://example.com/style.css | wc -c)
echo "Body bytes: $BYTES (expected 0)"
# Verify 200 returned with stale ETag
curl -sI -H 'If-None-Match: "stale"' https://example.com/style.css | head -1
# Expected: HTTP/2 200
```
### Verify open redirect protection
```bash
# Test that /redirect?url=... rejects external URLs
curl -sI "https://example.com/redirect?url=https://evil.com" | head -3
# Expected: 400 Bad Request OR 200 with safe behavior, NOT 302 to evil.com
# Test path only redirect
curl -sI "https://example.com/login?return_to=/dashboard" | head -3
# Expected: passthrough OR 302 to /dashboard
curl -sI "https://example.com/login?return_to=https://evil.com" | head -3
# Expected: rejected or sanitized
```
### Server side investigation
```bash
# Find all return 3xx in nginx config
nginx -T 2>/dev/null | grep -E "return 30[0-9]+"
# Show all server_name patterns
nginx -T 2>/dev/null | grep "server_name"
# Find any 302 that might need conversion to 301
nginx -T 2>/dev/null | grep -B 5 "return 302" | grep -E "location|server_name"
# Apply changes
nginx -t && systemctl reload nginx
```
### Browser DevTools quick reference
In Chrome DevTools Network panel:
1. Enable "Preserve log" to see redirects across navigation.
2. The redirect chain shows as multiple request rows.
3. Click each request to see Status code and Location header.
The Network panel's "Doc" filter shows just HTML/redirect requests, hiding asset noise.
For checking HSTS interaction:
1. Navigate to `chrome://net-internals/#hsts`.
2. Query the domain to see HSTS state.
3. "Delete domain" if testing rollback.
---
## 23. CROSS-REFERENCES
* [framework-http-caching-headers.md](framework-http-caching-headers.md): `Last-Modified` and `ETag` pair with conditional GET; 304 is the conditional success.
* [framework-http-seo-headers.md](framework-http-seo-headers.md): `Location` header is the redirect target; open redirect security pattern covered there is expanded in Section 15 here.
* [framework-http-security-headers.md](framework-http-security-headers.md): HSTS interaction (Section 14 here); HSTS forces HTTPS, making the HTTP to HTTPS redirect effectively permanent.
* [framework-http-request-headers.md](framework-http-request-headers.md): `If-Modified-Since` and `If-None-Match` request headers are what trigger 304 responses.
* [framework-http-2xx-status-codes.md](framework-http-2xx-status-codes.md): the previous status code framework; 201 Created with Location is analogous to 301 in that both use Location, but for different purposes.
* [UNIVERSAL-RANKING-FRAMEWORK.md](UNIVERSAL-RANKING-FRAMEWORK.md): the master ranking reference. 301 redirect equity transfer is fundamental to site migrations.
* [SEO-BUILD-REFERENCE.md](SEO-BUILD-REFERENCE.md) v2.4: the build playbook including the 14 paying client subdomain pattern.
* [framework-http-4xx-status-codes.md] (next in series): 400, 401, 403, 404, 410, 422, 429.
* [framework-http-5xx-status-codes.md] (later in series): 500, 502, 503, 504.
* Google HTTP status codes documentation: https://developers.google.com/search/docs/crawling-indexing/http-network-errors
* Google Change of Address tool documentation: https://support.google.com/webmasters/answer/9370220
* MDN 301: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/301
* MDN 302: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/302
* MDN 303: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/303
* MDN 304: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/304
* MDN 307: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/307
* MDN 308: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/308
* RFC 9110 (HTTP Semantics): https://www.rfc-editor.org/rfc/rfc9110
* RFC 7538 (308 Permanent Redirect): https://www.rfc-editor.org/rfc/rfc7538
---
## APPENDIX A: ONE PAGE QUICK REFERENCE
For the person who just wants the answer.
### Redirect decision matrix
```
Is the move permanent?
|
YES NO
| |
v v
Does method matter? Does method matter?
(POST stays POST?) (POST stays POST?)
| |
YES to 308 YES to 307
| |
NO to 301 NO to 302
Special cases:
* After POST, want GET response to 303
* Client has cached, version fresh to 304 (not really a redirect)
```
### SEO signal strength
| Redirect | SEO Signal | Use For |
|---|---|---|
| 301 | Strong (full equity transfer) | Permanent site/URL moves |
| 308 | Strong (same as 301) | Permanent API moves with method preservation |
| 302 | Weak (origin retained) | Login, A/B test, maintenance |
| 307 | Weak (same as 302) | Temporary API moves with method preservation |
| 303 | Not a canonical signal | Post/redirect/get pattern |
| 304 | Not a redirect | Conditional GET cache validation |
### Five rules to memorize
1. **301/308 for permanent**. 302/307 for temporary.
2. **307/308 preserve method**. 301/302 may change POST to GET.
3. **One hop only**. No chains.
4. **No open redirects**. Allow list or path only.
5. **HTTPS canonical in single hop from HTTP**. HSTS handles future visits.
### Five commands every operator should know
```bash
# 1. Check redirect status and destination
curl -sI https://example.com/old-url | head -3
# 2. Follow chain and count hops
curl -sIL https://example.com/old-url | grep -c "^HTTP"
# 3. Verify method preservation
curl -sIL -X POST https://example.com/api/v1/endpoint 2>&1 | grep -E "^> POST|^> GET|^HTTP"
# 4. Test conditional GET (304)
ETAG=$(curl -sI https://example.com/style.css | grep -i etag | awk '{print $2}' | tr -d '\r')
curl -sI -H "If-None-Match: $ETAG" https://example.com/style.css | head -1
# 5. Apply nginx changes
nginx -t && systemctl reload nginx
```
### Three end to end tests
```bash
# 1. HTTP to canonical HTTPS is exactly one hop
HOPS=$(curl -sIL http://example.com/ | grep -c "^HTTP")
[ "$HOPS" = "2" ] && echo "OK: one hop" || echo "FAIL: $((HOPS - 1)) hops"
# 2. No 302 for permanent canonicalization
nginx -T 2>/dev/null | grep -B 5 "return 302" | grep -iE "https|www|canonical" && echo "WARN: possible misuse of 302"
# 3. Method preserved through 308
curl -sIL -X POST -d "data=test" https://example.com/api/v1/users 2>&1 | \
grep -E "^> POST.*v2|^> GET.*v2" | head -1
# Expected: POST /api/v2/users (method preserved)
```
If all three pass AND no redirect chains exceed one hop, the 3xx layer is correctly wired.
---
End of framework-http-3xx-status-codes.md.