Menu

Engineering & CAD / Sample task

Generated public sample

Model a bracket from its drawing.

A complete drawing-to-model task. Every file is generated from one set of parameters; this is no company’s real part.

Compare submissions Read the drawing

Download the task folder

Gold output · drag to turnAmber: the bore C4 measures
The generated mounting bracket, shaded grey with dark edges: a base plate with four holes and a slot, an upright web with one bore shown in amber
Drag to rotate · use + / − to zoom · double-click to reset
Domain
Mechanical parts
Task family
Drawing to solid model
Access mode
Files, headless CAD kernel
Deliverable
One STEP solid
Verifier
Deterministic geometry: 3 gates, 10 checks
Time limit
45 minutes
Split
Public sample, never in a training or evaluation set

Reference drawing

DUF-S-001, rev A.

Three views in third-angle projection, generated from the same parameters as the gold model, so the drawing and the answer cannot drift apart. Amber marks the toleranced bore the verifier measures.

Reference drawing DUF-S-001 rev A: front, top and right views of an L-shaped mounting bracket, 120 by 80 by 84 millimetres, with four 9 millimetre through holes on a 90 by 36 pattern, a 10 by 30 slot, and an 18 millimetre H7 bore in the upright, drawn in amber
Generated drawing · open full size (SVG, 21.6 KB)

See it run

Compare the submissions.

Recorded verifier results. Yellow marks the failed checks; expand a report for every measurement.

Pick a submission

Submitted

output/bracket.step  ←  tests/gold/bracket.step
91,297 bytes · sha256 1785336204458725…
The gold solid the verifier compares against
gold solid: must pass strict · as expected, 0.87 s
output/bracket.step  ←  written by solution/solve.py (85,015 bytes, built in 0.61 s)
solution/solve.py: 7,551 bytes · sha256 ae73b38a9cc37870…
The reference solution: solution/solve.py builds the part from the drawing’s dimensions with OCP
reference solution: must pass strict · as expected, 0.91 s
output/bracket.step  ←  tests/probes/alt_rebuilt.step
83,731 bytes · sha256 9a4dc6d362af2d15…
Built another way: two outlines fused, one revolved bore cutter, fillet last; STEP AP242
correct alternative: must pass strict · as expected, 0.88 s
output/bracket.step  ←  tests/probes/alt_bore_18009.step
91,386 bytes · sha256 00e293e59fc76bb1…
The bore at Ø18.009, inside H7, everything else gold
correct alternative: must pass strict · as expected, 0.82 s
output/bracket.step  ←  tests/probes/alt_split_faces.step
118,131 bytes · sha256 69c196eb32cbaa4a…
The gold with every closed face split in two, as some CAD systems write it
correct alternative: must pass strict · as expected, 0.87 s
output/bracket.step  ←  tests/probes/empty.step
0 bytes · sha256 e3b0c44298fc1c14…
An empty bracket.step
reward hack: must score exactly 0 · as expected, 0.68 s
output/bracket.step  ←  a note, and no output/bracket.step
61 bytes · sha256 a1fbda2711798837…
No model, only a note saying the part was modelled and checked
reward hack: must score exactly 0 · as expected, 0.68 s
output/bracket.step  ←  tests/probes/scaled_copy.step
96,498 bytes · sha256 7a19df708266b18d…
The gold solid scaled by 1.02
plausible mistake: must fail strict, caught by C1 to C8 · as expected, 0.91 s
output/bracket.step  ←  tests/probes/bore_oversize.step
91,317 bytes · sha256 201c90aa78c85c88…
The bore 0.4 mm oversize, everything else gold
plausible mistake: must fail strict, caught by C4 · as expected, 0.82 s
output/bracket.step  ←  tests/probes/three_holes.step
86,917 bytes · sha256 d8f2fa15affd4f7e…
One base hole left out
plausible mistake: must fail strict, caught by C6, C7 · as expected, 0.88 s
output/bracket.step  ←  tests/probes/mirrored.step
89,522 bytes · sha256 28dcdc826bc8b551…
The gold solid mirrored left to right
plausible mistake: must fail strict, caught by C3, C8, C10 · as expected, 0.91 s
output/bracket.step  ←  tests/probes/spline_faces.step
135,874 bytes · sha256 5a1a0ede8035bf36…
The gold with every face converted to a B-spline surface: same shape, no true cylinders
plausible mistake: must fail strict, caught by C9 · as expected, 1.87 s

Verifier output · gold strict pass · reward 1 · score 1.00

All gates and checks passed.

Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 91297 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 209268.6 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 13 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass true
reward 1
score 1.00

Verifier output · solve_py strict pass · reward 1 · score 1.00

All gates and checks passed.

Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 85015 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 24 faces, volume 209265.5 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0090, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 12 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass true
reward 1
score 1.00

Verifier output · alt_rebuilt strict pass · reward 1 · score 1.00

All gates and checks passed.

Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 83731 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 24 faces, volume 209268.6 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 12 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass true
reward 1
score 1.00

Verifier output · alt_bore_18009 strict pass · reward 1 · score 1.00

All gates and checks passed.

Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 91386 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 209265.5 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0090, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 13 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass true
reward 1
score 1.00

Verifier output · alt_split_faces strict pass · reward 1 · score 1.00

All gates and checks passed.

Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 118131 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 32 faces, volume 209268.6 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 4 cone, 18 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass true
reward 1
score 1.00

Verifier output · empty strict fail · reward 0 · score 0.00

G1 FAIL output/bracket.step exists, is not empty and reads as STEP: the file is empty (0 bytes)
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 FAIL output/bracket.step exists, is not empty and reads as STEP: the file is empty (0 bytes)
checks  not scored: a gate failed
strict_pass false
reward 0
score 0.00

Verifier output · claim_only strict fail · reward 0 · score 0.00

G1 FAIL output/bracket.step exists, is not empty and reads as STEP: no file at output/bracket.step
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 FAIL output/bracket.step exists, is not empty and reads as STEP: no file at output/bracket.step
checks  not scored: a gate failed
strict_pass false
reward 0
score 0.00

Verifier output · scaled_copy strict fail · reward 0 · score 0.10

C1 FAIL Envelope: 122.400 × 81.600 × 85.680 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 FAIL Volume: 222.08 cm³ (+6.12%) expected 209.27 cm³ ±0.5% · w10
C3 FAIL Overlap with the gold (IoU): 0.8386 expected ≥ 0.995 · w10
C4 FAIL Bore diameter: Ø18.3600, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 FAIL Bore axis: position, square to B: X 61.200, Z 53.040; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 FAIL Through holes: count and diameter: 4 through, all Ø9.180 expected 4 × Ø9.0 ±0.10, through · w10
C7 FAIL Hole pattern: centre positions: 0 of 4 within 0.10, worst 2.100 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 FAIL Slot: width, length, centre: 10.200 × 30.600 at (42.840, 30.600) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 96498 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 222077.5 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 122.40 × 81.60 × 85.68 mm
C1 FAIL Envelope: 122.400 × 81.600 × 85.680 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 FAIL Volume: 222.08 cm³ (+6.12%) expected 209.27 cm³ ±0.5% · w10
C3 FAIL Overlap with the gold (IoU): 0.8386 expected ≥ 0.995 · w10
C4 FAIL Bore diameter: Ø18.3600, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 FAIL Bore axis: position, square to B: X 61.200, Z 53.040; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 FAIL Through holes: count and diameter: 4 through, all Ø9.180 expected 4 × Ø9.0 ±0.10, through · w10
C7 FAIL Hole pattern: centre positions: 0 of 4 within 0.10, worst 2.100 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 FAIL Slot: width, length, centre: 10.200 × 30.600 at (42.840, 30.600) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 13 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.84 < bore X 61.20: left expected slot left of the bore, seen from the front · w5
strict_pass false
reward 0
score 0.10

Verifier output · bore_oversize strict fail · reward 0 · score 0.80

C4 FAIL Bore diameter: Ø18.4000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 91317 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 209130.1 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.13 cm³ (-0.07%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 0.9993 expected ≥ 0.995 · w10
C4 FAIL Bore diameter: Ø18.4000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 13 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass false
reward 0
score 0.80

Verifier output · three_holes strict fail · reward 0 · score 0.80

C6 FAIL Through holes: count and diameter: 3 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 FAIL Hole pattern: centre positions: 3 of 4 within 0.10, worst 0.000, 1 missing expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 86917 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 24 faces, volume 210032.0 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 210.03 cm³ (+0.36%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 0.9964 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 FAIL Through holes: count and diameter: 3 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 FAIL Hole pattern: centre positions: 3 of 4 within 0.10, worst 0.000, 1 missing expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 12 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass false
reward 0
score 0.80

Verifier output · mirrored strict fail · reward 0 · score 0.75

C3 FAIL Overlap with the gold (IoU): 0.9686 expected ≥ 0.995 · w10
C8 FAIL Slot: width, length, centre: 10.000 × 30.000 at (78.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C10 FAIL Handedness: slot X 78.00 > bore X 60.00: right expected slot left of the bore, seen from the front · w5
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 89522 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 209268.6 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 FAIL Overlap with the gold (IoU): 0.9686 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 FAIL Slot: width, length, centre: 10.000 × 30.000 at (78.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 ok Feature types: all present · faces: 2 cone, 13 cylinder, 10 plane expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 FAIL Handedness: slot X 78.00 > bore X 60.00: right expected slot left of the bore, seen from the front · w5
strict_pass false
reward 0
score 0.75

Verifier output · spline_faces strict fail · reward 0 · score 0.95

C9 FAIL Feature types: missing: bore cylinder; 0/4 hole cylinders; R6 fillet ×0; R10 rounds ×0; 1×45° chamfers ×0 · faces: 25 B-spline expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
Full verifier output
$ bash tests/test.sh output/bracket.step
G1 ok output/bracket.step exists, is not empty and reads as STEP: 135874 bytes, read as STEP
G2 ok exactly one closed, valid solid with positive volume: 1 closed valid solid, 25 faces, volume 209268.6 mm³
G3 ok units are millimetres and the part fits inside a 250 mm cube: millimetre; extent 120.00 × 80.00 × 84.00 mm
C1 ok Envelope: 120.000 × 80.000 × 84.000 expected 120.0 × 80.0 × 84.0 ±0.10 · w10
C2 ok Volume: 209.27 cm³ (+0.00%) expected 209.27 cm³ ±0.5% · w10
C3 ok Overlap with the gold (IoU): 1.0000 expected ≥ 0.995 · w10
C4 ok Bore diameter: Ø18.0000, 3 sections within 0.0000 expected Ø18 H7: 18.000 to 18.018 · w20
C5 ok Bore axis: position, square to B: X 60.000, Z 52.000; square 0.0000 expected X 60.00, Z 52.00 ±0.05; square ≤ 0.02 · w10
C6 ok Through holes: count and diameter: 4 through, all Ø9.000 expected 4 × Ø9.0 ±0.10, through · w10
C7 ok Hole pattern: centre positions: 4 of 4 within 0.10, worst 0.000 expected 90.0 × 36.0 from (15, 14), ±0.10 · w10
C8 ok Slot: width, length, centre: 10.000 × 30.000 at (42.000, 30.000) expected 10.0 × 30.0 at (42, 30) ±0.10 · w10
C9 FAIL Feature types: missing: bore cylinder; 0/4 hole cylinders; R6 fillet ×0; R10 rounds ×0; 1×45° chamfers ×0 · faces: 25 B-spline expected bore and holes true cylinders; R6 fillet, 4 × R10, 2 × 1×45° chamfers · w5
C10 ok Handedness: slot X 42.00 < bore X 60.00: left expected slot left of the bore, seen from the front · w5
strict_pass false
reward 0
score 0.95

One run, 14 November 2026, in the task container, built from environment/Dockerfile and run with the network off (Docker 29.1.3, Linux): Python 3.12.13, OpenCascade through OCP 8.0.1.1, numpy 2.5.4. 12 submissions: the gold solid, the reference solution’s output and 10 probes (2 reward hacks, 5 plausible mistakes, 3 correct alternatives), each graded in its own process: 0.68 to 1.87 s apiece, 12.3 s in all. 12 of 12 as expected. Verifier sha256 ea5e7f60c430b8ea…

Task brief

What the agent is told.

The instruction supplied to the agent, alongside the files and tools it can access.

instruction.md818 bytes

# Model the bracket on drawing DUF-S-001

`project/drawing.svg` is a released drawing (rev A) of a mounting bracket. Model it as one solid and export it as STEP (AP214 or AP242) to `output/bracket.step`.

- Work in millimetres. Put the origin where datums A, B and C meet, with +X along the front edge, +Y towards the web and +Z up.
- Model every feature on the drawing: the four through holes, the slot, the bore with its chamfers, the rounds and the inner fillet.
- Model the Ø18 H7 bore at a diameter inside its tolerance (18.000 to 18.018).
- The drawing governs. Where it is silent, ISO 2768-m applies.
- Python 3.12 with OpenCascade (OCP, the kernel CadQuery is built on) and numpy is installed. You have no network access.

When the file is written, say so in one line. Do not describe checks you did not run.
The agent sees
project/drawing.svg and an empty output/ folder, in /workspace
Tools
A shell, Python 3.12 with OpenCascade (OCP) and numpy, as the Dockerfile installs them; no network
Hidden from it
The gold solid, the probes and the checks in tests/, copied in only after the run ends, and the reference solution in solution/
Variants
The generator takes every dimension, the hole pattern and the fit as parameters, so one family yields many distinct tasks

Deliverable and verifier

Graded on geometry, feature by feature.

Gates first: an answer that is missing, broken or in the wrong units scores 0. Then ten weighted checks, each a measurement with a stated tolerance. Whole-shape comparisons come first. Then every toleranced feature is measured on its own, in sections cut through the submitted solid, and its faces are checked for type.

File
output/bracket.step, STEP AP214 or AP242, exactly one solid
Frame
Millimetres, origin at datums A, B and C, Z up; the verifier also aligns on the datum faces before comparing
Not graded
Feature tree, body names, colours, layers, how the model was built
Reward
1 only if every gate and every one of the ten checks passes (strict pass), else 0. This is the training reward
Score
Sum of the passed checks’ weights, divided by 100; 0 if any gate fails. It shows where an answer went wrong

Gates

GateMust hold
G1output/bracket.step exists, is not empty and reads as STEP
G2It holds exactly one closed, valid solid with positive volume
G3Units are millimetres and the part fits inside a 250 mm cube

Checks

CheckWhat is measuredTargetToleranceWeight
C1Envelope120.0 × 80.0 × 84.0 mm±0.10 mm10
C2Volume209.27 cm³ (gold)±0.5%10
C3Overlap with the gold solid, aligned on A, B, CIntersection over unionat least 0.99510
C4Bore diameter, a circle fitted to the bore wall in three sectionsØ18 H718.000 to 18.018 mm20
C5Bore axis: position, and square to BX 60.00, Z 52.00±0.05 mm; 0.02 mm10
C6Through holes in the base: count and diameter4 × Ø9.0, through±0.10 mm10
C7Hole pattern: centre positions90.0 × 36.0, first hole at (15, 14)±0.10 mm10
C8Slot: width, length, centre10.0 × 30.0, centre (42, 30)±0.10 mm10
C9Feature types presentBore and holes true cylinders; R6 inner fillet, four R10 rounds, two 1 × 45° bore chamfersby face type; radii to ISO 2768-m5
C10HandednessSlot left of the bore, seen from the frontexact5

C4, in amber on the drawing and the model, is the check a whole-shape comparison cannot replace.

Probes

Wrong answers it must reject, right ones it must pass.

Before a task ships, its verifier is run on the reference solution’s output, on answers that are wrong on purpose and on correct alternatives built another way. Reward hacks must score exactly 0, every plausible mistake must fail strict on the checks it was built to trip, and the reference solution and every correct alternative must pass. Each row below is from the recorded run.

All probe submissions and measurements
SubmissionWhat it isKindMeasured by the verifierCaught byScoreStrict
goldThe gold solid the verifier compares againstgold solidenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0000Nothing1.00Pass
solve_pyThe reference solution: solution/solve.py builds the part from the drawing’s dimensions with OCPreference solutionenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0090Nothing1.00Pass
alt_rebuiltBuilt another way: two outlines fused, one revolved bore cutter, fillet last; STEP AP242correct alternativeenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0000Nothing1.00Pass
alt_bore_18009The bore at Ø18.009, inside H7, everything else goldcorrect alternativeenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0090Nothing1.00Pass
alt_split_facesThe gold with every closed face split in two, as some CAD systems write itcorrect alternativeenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0000Nothing1.00Pass
emptyAn empty bracket.stepreward hackThe file is empty (0 bytes)G10.00Fail
claim_onlyNo model, only a note saying the part was modelled and checkedreward hackNo file at output/bracket.stepG10.00Fail
scaled_copyThe gold solid scaled by 1.02plausible mistakeenvelope 122.400 × 81.600 × 85.680; volume +6.12%; overlap 0.8386; bore Ø18.3600C1 to C80.10Fail
bore_oversizeThe bore 0.4 mm oversize, everything else goldplausible mistakeenvelope within 0.10; volume -0.07%; overlap 0.9993; bore Ø18.4000C40.80Fail
three_holesOne base hole left outplausible mistakeenvelope within 0.10; volume +0.36%; overlap 0.9964; bore Ø18.0000; through holes 3 through, all Ø9.000; hole pattern 3 of 4 within 0.10, worst 0.000, 1 missingC6, C70.80Fail
mirroredThe gold solid mirrored left to rightplausible mistakeenvelope within 0.10; volume +0.00%; overlap 0.9686; bore Ø18.0000; slot 10.000 × 30.000 at (78.000, 30.000); handedness slot X 78.00 > bore X 60.00: rightC3, C8, C100.75Fail
spline_facesThe gold with every face converted to a B-spline surface: same shape, no true cylindersplausible mistakeenvelope within 0.10; volume +0.00%; overlap 1.0000; bore Ø18.0000; no true cylinder, cone or planeC90.95Fail

Generated from tests/run_results.json, the run in the next section. The oversize bore passes the envelope, the volume and the overlap and is caught only when the bore is measured, as in our own geometry study. The B-spline copy has the right shape and fails only on feature type: a CAM system or an inspector finds no true cylinder in it to pick up as a hole.

tests/verify.py · how C4 and C5 measure the bore
def measure_bore(solid):
    """The Ø18 H7 bore. Cut the web square to the bore at three depths, clear of the
    1 mm chamfers; fit a circle to the bore's outline in each section. The diameter is
    each circle's; the axis runs through the centres (position and squareness, C5)."""
    y0, y1 = web_faces(solid)                  # the web's front and back faces, measured
    ch = SPEC["bore"]["chamfer"]
    depths = (y0 + ch + 0.5, (y0 + y1) / 2, y1 - ch - 0.5)
    nominal = np.array([SPEC["bore"]["x"], SPEC["bore"]["z"]])
    rings = []
    for y in depths:
        loops = round_loops(section_loops(solid, "y", y))   # inner loops that are circles
        near = [l for l in loops if np.linalg.norm(l["centre"] - nominal) < 15]
        if not near:
            return None                        # no bore: C4 and C5 fail
        rings.append(min(near, key=lambda l: np.linalg.norm(l["centre"] - nominal)))
    shift = np.linalg.norm(rings[2]["centre"] - rings[0]["centre"])
    tilt = shift * (y1 - y0) / (depths[2] - depths[0])   # over the web's thickness
    return dict(d=[l["d"] for l in rings], centre=rings[1]["centre"], square=float(tilt),
                roundness=max(l["roundness"] for l in rings))

Measured on the part, not read from the file

The verifier never trusts a feature’s own label. It cuts the solid square to the bore at three depths clear of the chamfers and fits a circle to the bore’s outline in each. The diameter must sit inside the 18 µm H7 band in every section; the axis through the centres gives the position and the squareness.

Holes and the slot are measured the same way, in sections through the base near both faces and at mid-thickness, so a hole counts only if it goes through. Feature types (C9) are the one check read from the faces’ own definitions.

Task folder

What ships, file by file.

The download contains each file listed below, including the recorded run and the scripts needed to reproduce it.

PathWhat it isSize
task.tomlId, domain, access mode, time limit, split, container, verifier command, reward rule, solution command, rights1.1 KB
instruction.mdThe brief above818 B
environment/DockerfileThe agent’s container: Python 3.12, OpenCascade through OCP, numpy, the drawing; nothing fetched at run time1.0 KB
environment/project/drawing.svgThe reference drawing, rev A21.6 KB
tests/test.shRuns the verifier; writes reward 0 first, so a crash or a timeout still scores 0836 B
tests/verify.pyThe gates and the ten checks27.0 KB
tests/gold/bracket.stepThe gold solid, on the evaluator’s side91.3 KB
tests/probes/The probes: 8 STEP solids, an empty file, a claim-only note, and probes.json, which says what each submission must score796.1 KB
tests/run_probes.pyRuns the reference solution, then the verifier on its output, the gold solid and every probe, and checks each result5.1 KB
tests/run_results.jsonThe run shown on this page36.9 KB
solution/solve.shRuns solve.py and writes output/bracket.step398 B
solution/solve.pyThe reference solution: builds the bracket from the drawing’s dimensions with OCP, without reading the gold solid7.6 KB

Download the task folder ZIP, 177.3 KB · a public generated sample, not an evaluation item

The run above was made in the container the Dockerfile builds, with the network off: the reference solution ran first with tests/ absent, then the verifier was added and run.

Want tasks like this for your harness?

Tell us the capability you are training and the format your harness takes. See the Engineering & CAD data we build tasks from, and how we build environments.

Request a sample pack