Harden skills for production resilience (agent benchmark)

[jwitcoski]

Summary

I'm new to agent/AI coding and built a small Mapbox playground to learn how agents actually behave with mapbox-agent-skills — same four-pack idea as my MapTiler lab (no skill → skill → improved → upload). Great work getting the agent skills off the ground.

I graded the same 22 challenges three ways (harsh mode):

Set	Avg
No skill	~79%
Current skill	~90%
This revision	~100%

That run showed recurring failure modes. This PR hardens the skills below so agents get clearer guidance on those gaps.

Live grader: https://witcoskitech.com/mapbox-playground/agent-grader.html

Problems it fixes

1. Missing token guard — blank map when MAPBOX_ACCESS_TOKEN unset / placeholder
2. No map.on('error') — silent style/tile/token failures
3. No teardown — missing map.remove() on SPA/page cleanup
4. Layers lost on setStyle() — no style.load rebind (now under web-integration-patterns)
5. Geocode without proximity — ambiguous places resolve wrong (REST + debounce)
6. Fake Directions metrics — cosmetic "Len/Time" instead of route.distance / route.duration
7. Single-band isochrones — one contour when UX needs 15/30/60
8. Broken deck.gl CDN — jsDelivr +esm vs stable UMD / MapboxOverlay
9. Draw without draw.create — toolbar present, handler missing/broken
10. Constant extrusion height — not data-driven ['get', 'height']
11. Native gaps — bare iOS Map { } with no annotation; Android missing INTERNET

Files

• mapbox-token-security
• mapbox-web-performance-patterns
• mapbox-search-patterns
• mapbox-geospatial-operations
• mapbox-web-integration-patterns
• mapbox-data-visualization-patterns/references/3d-extrusions.md
• mapbox-ios-patterns
• mapbox-android-patterns

Test plan

• Review each added anti-pattern / checklist item in the files above
• Spot-check agents following the updated skills: token guard before new Map, map.on('error'), style.load after setStyle
• Confirm Directions/Isochrone and deck.gl guidance match production CDN/API usage

[underoot]

Seems not related to style patterns?

[jwitcoski]

Agreed — style-patterns is recipe/layer config, not map lifecycle. Moved the setStyle / style.load guidance into mapbox-web-integration-patterns as Mistake 8.

[mattpodwysocki]

Thanks for putting this together, and for including the grader numbers, that's a great way to show the impact of these changes. The anti-patterns you're targeting are real and worth documenting (token guard, error handling, style.load rebind, draw.create, data-driven extrusion height, INTERNET permission). This is good work for someone new to agent skills.

Before we merge, a few things need fixing:

1. The token guard snippet in mapbox-token-security/SKILL.md chains window.MAPBOX_ACCESS_TOKEN || import.meta.env.VITE_MAPBOX_ACCESS_TOKEN || process.env.NEXT_PUBLIC_MAPBOX_ACCESS_TOKEN in one example. That doesn't actually work in a single environment. In a Vite app without the bundler, process isn't defined in the browser, so that line throws a ReferenceError before your guard even runs. In a plain script tag (which is exactly the context some of your other examples use, like the deck.gl CDN snippet), import.meta.env access will throw too. Every other example in this repo scopes to one bundler at a time and points to references/token-management.md for the others. Can you follow that same pattern here?

2. mapbox-data-visualization-patterns/references/3d-extrusions.md is missing a trailing newline and the new heading jumps from h1 straight to h3. Both fail our markdownlint and format:check steps in CI.

3. Spellcheck is failing on two words: "racey" in mapbox-search-patterns/SKILL.md (typo for "racy") and "Scatterplot" in mapbox-web-integration-patterns/SKILL.md (from ScatterplotLayer, needs to go in cspell.config.json or get reworded).

Can you run npm run format, npm run lint:markdown, and npm run spellcheck locally and push fixes? Once those are clean this should be in good shape. The content itself is solid, just need to get it past our checks.

[mattpodwysocki]

Thanks for the fast turnaround, the token guard, heading fix, and spellcheck fixes all look correct, I verified them locally against your latest commits.

CI just ran for the first time on this PR and it's failing on format:check:

[warn] skills/mapbox-geospatial-operations/SKILL.md

The new Directions example doesn't match Prettier's line wrapping:

stats.textContent =
  `Distance ${(route.distance / 1000).toFixed(2)} km · ` +
  `Duration ${Math.round(route.duration / 60)} min`;

Prettier wants that on one line given our 120 char width for markdown. Can you run npm run format locally and push the result? That should be the only thing left blocking green CI.

[jwitcoski]

Thanks... glad the token guard, heading, spellcheck, and Prettier wrap all checked out. This was fun, and I appreciate you getting back so quickly with concrete fixes.

I'm relatively new to getting code reviewed (and to AI coding), so the feedback was really useful.

I'm also hoping to move the AI map / skill-agent grading lab onto its own site: map-agent-arena.Live view Still a few days of work, but the idea is to run ~11 frontend map vendors' agents through the same map challenges and show the results side by side.

Thanks again for the review.

[mattpodwysocki]

Verified at HEAD 15926f4: token guard fix, heading/newline fix, spellcheck fixes, and the Prettier wrap all check out. Re-ran the full `npm run check` suite in an isolated worktree at this exact commit and it's clean. CI is green. Approving, thanks for the quick fixes.