RampNet 2.0: measurement, condition inference, and tagging — we already collect the supervision and discard it at ingest

[jonfroehlich]

The thesis

RampNet 2.0 is not "find more curb ramps." It is find + measure + rate + tag — moving from "is there a ramp here?" to "is this ramp usable, and what is wrong with it?"

Detection is the part that is close to done. RampNet holds F1 0.908 on un-anchored gold GT and beats all eight challengers on every benchmark split by 0.19–0.34 F1; the remaining levers are incremental (#54's threshold change bought +7.7 recall for free, #78's TTA bought nothing). The marginal value of a better detector is falling, and the north star — an AI labeler that finds, tags and rates at least as well as humans — is three-quarters unstarted.

And this is the part that street-level imagery uniquely owns. #85 established that aerial orthoimagery already detects curb ramps in production (34k ramps at 1-inch GSD). What aerial structurally cannot do is judge slope, surface condition, obstruction, tactile warning, or whether a ramp points into traffic — all of which need a pedestrian-height viewpoint. Condition inference is therefore the durable differentiator, not a nice-to-have.

The unique data we already have and currently discard

This is the concrete finding. Three separate streams of condition and measurement supervision already flow through this repository, and all three are truncated to geometry at read time:

1. Project Sidewalk severity, tags, and descriptions — dropped on ingest.
stage_one/crop_model/ps_model/data/download_data.py pulls /v2/rawLabels CSV exports from 12 Project Sidewalk city deployments (blackhawk-hills, chicago, cliffside-park, columbus, knox, mendota, newberg, oradell, pittsburgh, sea, st-louis, teaneck) and consumes exactly six columns: Label Type, Agree Count, Disagree Count, Panorama ID, Panorama X/Y, Panorama Width/Height (lines 237–262). Every label's severity rating (1–5), tags, and free-text description are read and thrown away.

That is the crown-jewel dataset for this problem — human accessibility judgments on curb ramps, at scale, with per-label validation counts — and the pipeline uses it purely as a source of pixel coordinates.

2. Government attributes — reduced to three fields.
stage_one/dataset_generation/combine_location_data.py writes all_locations.csv with exactly latitude, longitude, date (line 95). Whatever the source geojson/csv carried — condition, compliance status, slope, width, material, ADA/PROWAG status, remediation history — is discarded at parse time. Municipal ramp inventories frequently carry several of these.

3. Extent — present in the gold set, unused.
manual_labels/*.txt are YOLO-format boxes (class cx cy w h); we have only ever consumed the centers. Noted already in #83, repeated here because it is the same pattern.

What the supervision actually supports

capability	supervision	notes
Tagging	PS tags, multi-label	Vocabulary includes Narrow (<3 ft), Points into cross traffic, Surface problem, Missing tactile warning, and others. Validation counts give a quality filter — the same Agree − Disagree ≥ 2 gate already used for detection
Rating	PS severity 1–5 (ordinal)	5 = impassable barrier for a wheelchair user
Measurement	derived, not annotated	Metric width, slope, landing size — quantities PROWAG actually specifies

The three interlock in a useful way: "Narrow (<3 ft)" is a measurement threshold, not an opinion. So a metric width estimator both predicts that tag and produces a continuous quantity, and the tag becomes a free validation signal for the measurement head.

And the measurement method is already demonstrated on our exact imagery. UrbanVGGT (March 2026) estimates metric sidewalk width from Google Street View — semantic segmentation + feed-forward 3D reconstruction + adaptive ground-plane fitting + camera-height scale calibration — at MAE 0.252 m, 95.5% within 0.50 m. Its ablation finds scale calibration is the critical component, which is the same 2.5 m camera-height trick already in prepare_yolo_dataset.py's pitch strategy. Ground-plane fitting is better conditioned for curb ramps than for the sidewalks it was built for, because a ramp is a ground-plane object by definition. See the #48 prior-art comment for the fuller stack.

Four things that will be genuinely hard — worth stating before anyone designs a model

1. Absence of a tag is not a negative. Most curb ramp labels almost certainly carry no tags at all. Treating untagged labels as negatives makes this a positive-unlabeled problem wearing a multi-label costume, and naive training will learn tag prevalence rather than tag presence. This is the trap most likely to produce a confidently wrong result.
2. Severity is subjective and rater-dependent. Second rater for Budapest District V: the rubric is now confounded with the benchmark's only ranking inversion #74 already showed rubric confounding on Budapest at low reviewer confidence — and that was a single expert rater on a binary-ish judgment. A 1–5 scale across many volunteers and 12 cities will carry substantial and structured noise. Inter-rater analysis has to come before treating severity as ground truth.
3. Temporal mismatch. PS labels and GSV panoramas both have dates, and condition changes. A 2019 severity rating against a 2023 panorama is measuring a different ramp. This confound does not exist for detection (ramps rarely move) but is central for condition.
4. Extent is a prerequisite for crop-based tagging. PS labels are points; a tagging model needs either a resolved ramp extent (Extend RampNet output from points to boxes/polygons (point-prompted SAM2; CenterNet-style size head; segmentation) #83) or a fixed crop convention, and the choice will materially affect results.

Proposed first step: a data audit, before any modelling

Cheap, CPU-only, no GPU, and decisive. Pull rawLabels for the 12 cities already in the pipeline and report:

• how many CurbRamp labels carry a severity rating, and the severity distribution per city;
• how many carry ≥1 tag, the tag frequency distribution, and the co-occurrence structure;
• the validation-count distribution (how many survive an Agree − Disagree ≥ 2 style gate with severity/tags attached);
• the distribution of the gap between label date and panorama capture date;
• inter-city variation in all of the above — a proxy for rater-population effects.

That answers "is this supervision dense and clean enough to train on?" before anyone commits GPU time or designs an architecture, and it is committable as a finding either way. If severity turns out to be sparse or wildly inconsistent across cities, that reshapes the whole plan — and is exactly the kind of negative result this repo exists to record.

A second, equally cheap audit: check what attributes the government files in location_data/ actually carry beyond lat/lng/date, since combine_location_data.py discards them sight-unseen.

Structure

Suggest treating this as the umbrella issue for the RampNet 2.0 condition/measurement scope, with the audit above as the first concrete task and separate sub-issues per track (measurement, tagging, severity) once the audit says which are viable.

Related

• Aerial orthoimagery curb-ramp detection already exists in production (34k ramps at 1-inch GSD): treat it as an uncorrelated arbiter, not a competitor #85 — aerial detects but structurally cannot rate; this is the capability that justifies street-level
• Extend RampNet output from points to boxes/polygons (point-prompted SAM2; CenterNet-style size head; segmentation) #83 — extent/polygons, a prerequisite for crop-based tagging and for measurement
• Multi-capture / multi-view fusion for per-ramp recall: correspondence, agreement-as-arbiter, 3D (RampNet 2.0) #48 — multi-view fusion; condition inference benefits from multiple views at least as much as detection does
• Stage 2 trains for 1 epoch at constant LR (~9.4k steps): measure the epoch curve, and with it the auto-label ceiling #84 — training recipe; any new head (severity, tags, size) is a Stage 2 retrain and inherits that budget question
• Second rater for Budapest District V: the rubric is now confounded with the benchmark's only ranking inversion #74 — the rater-subjectivity problem, already observed on a much simpler judgment

Note on sourcing: the Project Sidewalk API and labeling-guide pages redirect-loop against automated fetching, so the tag vocabulary above is partial and drawn from secondary sources. The authoritative list should be pulled from the API during the audit.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

Mensuration feasibility: what the geometry actually permits

Working through whether measurement is achievable from GSV panoramas via modern depth models or multi-view. Short version: the primary method should not be monocular depth inference, geometry is not the binding constraint for slope (I initially thought it was), and the real limiters are correspondence, pose, and panorama stitching — which have to be measured, not derived.

All derivations below are mine, with assumptions stated so they can be checked. Citations are marked as such.

Setup

• Camera height ≈ 2.5 m (the constant already used by prepare_yolo_dataset.py's pitch box-size strategy)
• Ramps at 5–25 m from the capture vehicle
• Our Stage 2 panoramas: 4096 px for 360° → 1.534 mrad/px
• GSV serves 16384 px at zoom 5 → 0.383 mrad/px

We currently throw away 4× angular resolution. That is correct for detection and expensive for mensuration. Mensuration should re-fetch native tiles at the detected ramp location rather than reuse the training-resolution panorama — a free 4× precision gain.

Method: ray–ground-plane intersection, not depth inference

For a ground-plane object with known camera height, position follows from ray–plane intersection with no depth model at all: d = h / tan θ. Propagating angular error, σ_d = σ_θ · (h² + d²) / h:

distance	our 4096 pano	native zoom 5
10 m	±6.5 cm	±1.6 cm
20 m	±24.9 cm	±6.2 cm

Consistency check: that ±24.9 cm at 20 m reproduces UrbanVGGT's reported MAE of 0.252 m almost exactly, which suggests their error is geometry-dominated — and that native resolution plus nearer ranges should beat it substantially.

By contrast, the best outdoor monocular metric depth results in recent benchmarks are ~0.45 m MAE (Depth Anything V2). Depth models are the wrong primary tool here: use them for the plane-fit prior and occlusion reasoning, not for the measurement.

Corollary — treat the ramp as the residual. A ramp is by definition the deviation from the surrounding ground plane. Fit the plane from adjacent sidewalk and street, then measure the ramp as the residual. That is better conditioned than measuring the ramp surface absolutely, and it is naturally robust to camera-height error, which perturbs scale rather than residual shape.

Lateral measurements — well conditioned

Width is d · Δθ, so depth error propagates proportionally, not additively. At 10 m the geometric contribution is sub-centimetre; edge localisation on concrete (±2–3 px) dominates at roughly ±3–5 cm. Against PROWAG's 48″ (122 cm) minimum width and 48″×48″ landing, that is decisive. Ramp footprint and landing dimensions fall out of the same fit.

Vertical / slope — geometry permits more than I first thought

My first pass had the direction cosine inverted and concluded slope was hopeless. Corrected:

Triangulating from consecutive panoramas (baseline ≈ 10 m), range error is σ_r = z²σ_θ/B ≈ 1.53 cm at 10 m (our resolution). The ray sits ~14° below horizontal at that range, so decomposing into vertical error:

σ_y = sqrt( (σ_r·sin θ)²  +  (r·σ_θ·cos θ)² )
    = sqrt( (1.53×0.24)² + (1.53×0.97)² )  ≈ 1.5 cm   (4096 px)
                                            ≈ 0.38 cm  (native zoom 5)

Over a 1.22 m ramp with uncorrelated endpoints, that is σ_slope ≈ 1.7% (our resolution) or ≈ 0.44% (native) — and 0.44% would in principle be enough to adjudicate the 8.3% running-slope threshold.

So geometry is not what rules slope out. What does the ruling is everything the idealisation assumes away:

1. Correspondence on textureless concrete. Flat grey concrete is the classic multi-view stereo failure case. You cannot localise a surface point to 1 px on it; dense MVS will smooth precisely the residual we are trying to measure. This is the dominant unknown.
2. Relative pose error. A 0.1° relative orientation error is ~1.75 mrad — larger than a full pixel. Translation error over a 10 m baseline maps straight into depth error.
3. Panorama stitching. GSV panos are stitched from multiple cameras on a moving vehicle. The result is not a perfect central projection, and that error is not modelled by any of the above.
4. Foreshortening. At 14° depression the ramp surface is seen nearly edge-on, which is the worst case for recovering its slope.

None of these can be quantified from first principles. Slope feasibility is therefore an empirical question, not a settled one — which is a meaningfully different conclusion from "out of reach."

Outside view, as a caution: the LiDAR literature on ADA curb-ramp assessment reports that consumer-grade "pocket" LiDAR — millimetre ranging at 1–2 m standoff — is useful only for sorting ramps clearly within or outside spec, with survey-grade systems and rigorous procedures required for actual compliance calls. That is a sobering calibration for photogrammetry at 10–25 m, and argues for framing any slope output as triage rather than compliance.

Cross slope and lip height — out of reach

PROWAG's 2% max cross slope is 2.4 cm of fall across a 1.22 m ramp, at or below the best-case vertical resolution before any of the four error sources above. The ¼″ vertical-discontinuity limit is two orders of magnitude out. These require LiDAR, and per the citation above, survey-grade LiDAR at that.

Proposed first experiment: cross-view repeatability, no ground truth needed

The binding question is precision on real imagery, and precision can be measured without any ground truth: measure the same ramp from N different panoramas and report the spread.

• If width measurements of one ramp scatter by ±5 cm across views, the lateral capability is real.
• If they scatter by ±40 cm, we are done, and no ground-truth campaign was needed to find out.
• The same procedure on the slope residual directly bounds whether slope triage is viable.

This is cheap, needs no survey data or field visit, and isolates exactly the error sources the derivation cannot predict. It also composes with #48 — cross-view repeatability is a multi-view correspondence experiment, so the two share the same plumbing.

Second, cheaper still: run the ray–plane width estimate at both 4096 and native resolution on the same ramps and confirm the predicted 4× improvement actually materialises. If it does not, the error budget is dominated by something other than angular resolution, and that tells us where to look.

One catch for the tagging head

Project Sidewalk's "Narrow" tag is defined as <3 ft (36″), but PROWAG specifies 48″ minimum width. The tag vocabulary encodes the older ADAAG figure. A width head trained purely against that tag learns a threshold that no longer matches the standard — whereas a metric width estimate produces both judgments, which is strictly more useful than the tag and is an argument for measurement over classification wherever the underlying quantity is continuous.

Summary

target	verdict
ramp width, landing dimensions, footprint	feasible — lateral, well conditioned, ±3–5 cm at 10 m
detectable-warning presence / coverage	feasible — a detection problem at native resolution, not mensuration
running slope	empirical — geometry permits ~0.4–1.7%, real limit set by correspondence/pose/stitching; frame as triage
cross slope (2%), lip height (¼″)	out of reach from street-level imagery

None of the feasible column is available from aerial (#85), which is the point.

🤖 Generated with Claude Code (claude-opus-5[1m])