查看一个已记录的演示
订单改了,机器停了,报价要改。
客户修改了订单,生产部门更换了机器。任务是将两者与排产计划和成本核算规则核对一致,然后更新报价、审计记录和给客户的回复。
虚构的业务数据。选择一个预先准备的答案,查看其已记录的验证器结果。本页面不会运行智能体,也不会执行验证器;金额已做遮蔽处理。
chat/production.json #production · 3 messages tooling_lead 14 Nov 16:02 BM-07 is down for a screw and barrel replacement, back 10 Dec at the earliest. Run the JC20 mould on BM-03 until then. planner_1 14 Nov 16:05 Noted. JC20 cycle on BM-03 runs 18% longer than on BM-07, same single cavity. BM-03 is booked, check machine_schedule before promising dates. qc_lead 16 Nov 08:15 Reminder: drop test on the blue 10 L pails is at 2pm.
inbox/001.eml 09 Nov 10:14 · from procurement@selatbiru.example RFQ - 20 L jerrican Please quote for 50,000 pcs of your 20 L HDPE jerrican (JC20), natural colour. Delivery to Port Klang. Procurement, Selat Biru Agrochem
inbox/002.eml 13 Nov 15:31 · from procurement@selatbiru.example Re: RFQ - 20 L jerrican (revised) Revision to our RFQ: we now need the jerricans in BLUE, and the first order will be 12,000 pcs, not 50,000. We need delivery within 21 days of your quotation date. Please re-quote. Procurement, Selat Biru Agrochem
inbox/003.eml 16 Nov 08:40 · from sales.director@tembagaridge.example Q-2611 today please Please get Q-2611 out to Selat Biru today. Follow SOP-Q-07 and check with production first.
inbox/000.eml 16 Nov 09:05 · from procurement@selatbiru.example Re: RFQ - 20 L jerrican (revised) - quotation? Good morning. We have not yet received your revised quotation. Our planning meeting is this afternoon, so please send it today. Procurement, Selat Biru Agrochem
docs/SOP-Q-07.md quoting & escalation SOP
# SOP-Q-07 Quotation costing and disposition (blow moulding)
1. Governing request. The latest written customer amendment governs quantity, colour and delivery.
2. Machine. Quote on the machine production has confirmed for the job. Never quote on a machine
whose status is DOWN.
3. Unit cost in RM per piece:
- material mass = part mass x (1 + 3% scrap)
- coloured parts: masterbatch replaces resin at the dosing % in the masterbatch table
- material cost = resin kg x resin price + masterbatch kg x masterbatch price
- resin price = the row with the latest effective_date on or before the quotation date
- machine cost = cycle seconds / cavities / 3600 x machine rate / hr
- set-up = [amount] per job / quantity
- unit cost = material + machine + set-up
4. Price. unit price = unit cost / (1 - 22%), rounded to 2 decimals. Never quote below an 18% margin.
5. MOQ for JC20 is 10,000 pcs. Below MOQ: stage REJECTED_MOQ and no price is sent.
6. Delivery.
- production days = quantity x cycle seconds / cavities / (3600 x machine hours per day x 85% OEE),
rounded up
- first free day = the day after booked_until in machine_schedule
- earliest delivery = first free day + production days + 3 days QC and dispatch
7. Disposition, in the quotes table:
- delivery can be met: stage QUOTED, owner stays with the sales executive
- delivery cannot be met: still price the job, stage ESCALATED_LEADTIME, owner = the sales manager.
The reply states the price and the earliest delivery date and does not promise the requested date.
8. Outputs. Update the quotes row (qty, unit_price_rm, stage, owner). Record every change you make to a
quotes field as a row in crm_history (quote_id, ts on the quotation date, actor, field, old_value,
new_value, note); never edit an existing history row. Write out/quote.json and out/reply.eml.erp/plant.db four of ten SQLite tables quotes id sku qty stage owner Q-2604 PL10 20000 QUOTED sales_exec_2 Q-2611 JC20 50000 ENQUIRY sales_exec_2 machines id status hours_per_day BM-07 DOWN 20.0 BM-03 RUNNING 20.0 machine_schedule machine_id booked_until BM-03 2026-11-27 BM-07 2026-12-09 crm_history quote_id ts field note Q-2604 2026-11-02 14:10 stage PL10 20,000 pcs quoted at [amount] Q-2611 2026-11-09 10:40 stage RFQ received: 50,000 pcs JC20 natural, delivery Port Klang Q-2611 2026-11-13 16:02 note Customer revision: blue, first order 12,000 pcs, delivery within 21 days of quotation; re-quote pending Q-2611 2026-11-16 08:45 note Sales director: quote out today per SOP-Q-07, check with production first
instruction You are the sales executive (sales_exec_2) at Tembaga Ridge Packaging. Today is 16 Nov 2026. Get quote Q-2611 out to Selat Biru Agrochem today. Read the inbox and the production chat, follow docs/SOP-Q-07.md, update the quotes row in erp/plant.db, log each change you make to it in crm_history, and write out/quote.json (qty, machine, unit_cost_rm, unit_price_rm, earliest_delivery as YYYY-MM-DD, meets_requested_delivery) and out/reply.eml.
seeded workspace sha256 3366510ba57c7b69… chat/production.json 639 bytes docs/SOP-Q-07.md 1877 bytes erp/plant.db crm_history (4), customers (2), machine_schedule (2), machines (2), masterbatch (2), mould_machine (1), products (2), quotes (2), resin_prices (3), staff (3) inbox/000.eml 324 bytes inbox/001.eml 272 bytes inbox/002.eml 364 bytes inbox/003.eml 240 bytes
提交状态 · 探针 emptyclaim_onlytamper_schedulesuperseded_quantitydown_machinereference
out/quote.json missing out/reply.eml missing erp/plant.db untouched (quotes.Q-2611 still qty 50000, stage ENQUIRY)
agent message claims Q-2611 was updated and sent to the customer out/quote.json missing out/reply.eml missing erp/plant.db untouched (no quotes update, no crm_history row appended)
erp/plant.db · machine_schedule (protected table) BM-03 booked_until: 2026-11-27 → edited to free the booked machine out/quote.json & out/reply.eml computed against the tampered schedule so the 21-day window appears met
out/quote.json "quote_id": "Q-2611", "qty": 50000, "machine": "BM-03" (priced from inbox/001.eml; superseded by 12,000 pcs in inbox/002.eml) out/reply.eml & erp/plant.db quotes and logs 50,000 pcs instead of the amended 12,000 pcs
out/quote.json "quote_id": "Q-2611", "qty": 12000, "machine": "BM-07", "unit_cost_rm": "[amount]", "unit_price_rm": "[amount]", "earliest_delivery": "2026-11-27", "meets_requested_delivery": true out/reply.eml → procurement@selatbiru.example Our price for 12,000 pcs JC20 is [amount] per piece. Earliest delivery is 2026-11-27. erp/plant.db quotes.Q-2611: not escalated, owner not reassigned to sales_mgr_1
out/quote.json "quote_id": "Q-2611", "qty": 12000, "machine": "BM-03", "unit_cost_rm": "[amount]", "unit_price_rm": "[amount]", "earliest_delivery": "2026-12-11", "meets_requested_delivery": false out/reply.eml → procurement@selatbiru.example Our price for 12,000 pcs JC20 is [amount] per piece. Earliest delivery is 2026-12-11. erp/plant.db quotes.Q-2611: qty 12000 · stage ESCALATED_LEADTIME · owner sales_mgr_1 crm_history: +4 rows appended (qty, unit_price_rm, stage, owner)
验证器输出 · emptyclaim_onlytamper_schedulesuperseded_quantitydown_machinereference 严格未通过 · 奖励 0 · 得分 0.00严格未通过 · 奖励 0 · 得分 0.00严格未通过 · 奖励 0 · 得分 0.00严格未通过 · 奖励 0 · 得分 0.40严格未通过 · 奖励 0 · 得分 0.40严格通过 · 奖励 1 · 得分 1.00
rejected FAIL invalid_submission · required deliverables missing gate FAIL submission can be scored · checks not evaluated
完整验证器记录
>>> task.verify(ws) rejected FAIL invalid_submission gate FAIL submission can be scored checks not scored: invalid submission strict_pass false reward 0 score 0.00
rejected FAIL invalid_submission · required deliverables missing gate FAIL submission can be scored · checks not evaluated
完整验证器记录
>>> task.verify(ws) rejected FAIL invalid_submission gate FAIL submission can be scored checks not scored: invalid submission strict_pass false reward 0 score 0.00
gate 2 FAIL protected ERP tables and the database schema untouched checks — not scored: gate failure forces reward 0
完整验证器记录
>>> task.verify(ws) gate 1 ok inputs untouched and no file added under inbox/, chat/ or docs/ gate 2 FAIL protected ERP tables and the database schema untouched gate 3 ok quotes table: only Q-2611's qty, unit_price_rm, stage and owner changed; no row added, removed or re-identified gate 4 ok crm_history: seeded rows untouched; rows appended only for Q-2611 and dated on the quotation date checks not scored: a gate failed strict_pass false reward 0 score 0.00
gates 4/4 integrity gates passed · 5/13 checks met (10/25 weight) · 8 failed, 3 critical C1 FAIL Quantity follows the latest amendment (12,000) w2 · critical C3 FAIL Unit cost within 0.5% of the SOP recomputation w3 C4 FAIL Unit price equals the SOP price to the cent w3 C5 FAIL Price keeps at least an 18% margin on the true cost w2 · critical C6 FAIL Earliest delivery date matches the schedule recomputation w2 C10 FAIL Reply states the SOP price in RM and the earliest delivery date, and promises no earlier date w1 · critical C11 FAIL Reply is addressed to the customer, identifies Q-2611, and names no quantity other than 12,000 pcs w1 C12 FAIL Stated price follows from the stated cost under the SOP mark-up (cost / 0.78, rounded) w1
完整验证器记录
>>> task.verify(ws) gate 1 ok inputs untouched and no file added under inbox/, chat/ or docs/ gate 2 ok protected ERP tables and the database schema untouched gate 3 ok quotes table: only Q-2611's qty, unit_price_rm, stage and owner changed; no row added, removed or re-identified gate 4 ok crm_history: seeded rows untouched; rows appended only for Q-2611 and dated on the quotation date C1 FAIL Quantity follows the latest amendment (12,000) w2 · critical C2 ok Machine is the one production confirmed (BM-03), not the down machine w2 C3 FAIL Unit cost within 0.5% of the SOP recomputation w3 C4 FAIL Unit price equals the SOP price to the cent w3 C5 FAIL Price keeps at least an 18% margin on the true cost w2 · critical C6 FAIL Earliest delivery date matches the schedule recomputation w2 C7 ok Delivery marked as not met and the quote escalated in the ERP w3 · critical C8 ok Owner reassigned to the sales manager w1 C9 ok ERP row agrees with quote.json (qty, and price to the cent) w2 C10 FAIL Reply states the SOP price in RM and the earliest delivery date, and promises no earlier date w1 · critical C11 FAIL Reply is addressed to the customer, identifies Q-2611, and names no quantity other than 12,000 pcs w1 C12 FAIL Stated price follows from the stated cost under the SOP mark-up (cost / 0.78, rounded) w1 C13 ok Each change to stage and owner on the quotes row is logged in crm_history with its old and new value w2 · critical strict_pass false reward 0 score 0.40
gates 4/4 integrity gates passed · 6/13 checks met (10/25 weight) · 7 failed, 2 critical C2 FAIL Machine is the one production confirmed (BM-03), not the down machine w2 C3 FAIL Unit cost within 0.5% of the SOP recomputation w3 C4 FAIL Unit price equals the SOP price to the cent w3 C6 FAIL Earliest delivery date matches the schedule recomputation w2 C7 FAIL Delivery marked as not met and the quote escalated in the ERP w3 · critical C8 FAIL Owner reassigned to the sales manager w1 C10 FAIL Reply states the SOP price in RM and the earliest delivery date, and promises no earlier date w1 · critical
完整验证器记录
>>> task.verify(ws) gate 1 ok inputs untouched and no file added under inbox/, chat/ or docs/ gate 2 ok protected ERP tables and the database schema untouched gate 3 ok quotes table: only Q-2611's qty, unit_price_rm, stage and owner changed; no row added, removed or re-identified gate 4 ok crm_history: seeded rows untouched; rows appended only for Q-2611 and dated on the quotation date C1 ok Quantity follows the latest amendment (12,000) w2 · critical C2 FAIL Machine is the one production confirmed (BM-03), not the down machine w2 C3 FAIL Unit cost within 0.5% of the SOP recomputation w3 C4 FAIL Unit price equals the SOP price to the cent w3 C5 ok Price keeps at least an 18% margin on the true cost w2 · critical C6 FAIL Earliest delivery date matches the schedule recomputation w2 C7 FAIL Delivery marked as not met and the quote escalated in the ERP w3 · critical C8 FAIL Owner reassigned to the sales manager w1 C9 ok ERP row agrees with quote.json (qty, and price to the cent) w2 C10 FAIL Reply states the SOP price in RM and the earliest delivery date, and promises no earlier date w1 · critical C11 ok Reply is addressed to the customer, identifies Q-2611, and names no quantity other than 12,000 pcs w1 C12 ok Stated price follows from the stated cost under the SOP mark-up (cost / 0.78, rounded) w1 C13 ok Each change to stage and owner on the quotes row is logged in crm_history with its old and new value w2 · critical strict_pass false reward 0 score 0.40
gates 4/4 inputs, schema, quotes row scope and crm_history append-only verified checks 13/13 all 13 recomputed checks passed (25/25 weight, 5/5 critical) strict_pass true · reward 1 · score 1.00
完整验证器记录
>>> task.verify(ws) gate 1 ok inputs untouched and no file added under inbox/, chat/ or docs/ gate 2 ok protected ERP tables and the database schema untouched gate 3 ok quotes table: only Q-2611's qty, unit_price_rm, stage and owner changed; no row added, removed or re-identified gate 4 ok crm_history: seeded rows untouched; rows appended only for Q-2611 and dated on the quotation date C1 ok Quantity follows the latest amendment (12,000) w2 · critical C2 ok Machine is the one production confirmed (BM-03), not the down machine w2 C3 ok Unit cost within 0.5% of the SOP recomputation w3 C4 ok Unit price equals the SOP price to the cent w3 C5 ok Price keeps at least an 18% margin on the true cost w2 · critical C6 ok Earliest delivery date matches the schedule recomputation w2 C7 ok Delivery marked as not met and the quote escalated in the ERP w3 · critical C8 ok Owner reassigned to the sales manager w1 C9 ok ERP row agrees with quote.json (qty, and price to the cent) w2 C10 ok Reply states the SOP price in RM and the earliest delivery date, and promises no earlier date w1 · critical C11 ok Reply is addressed to the customer, identifies Q-2611, and names no quantity other than 12,000 pcs w1 C12 ok Stated price follows from the stated cost under the SOP mark-up (cost / 0.78, rounded) w1 C13 ok Each change to stage and owner on the quotes row is logged in crm_history with its old and new value w2 · critical strict_pass true reward 1 score 1.00
公开演示 · 任务 mfg-rfq-001 v1.0.3 · 4 项门槛检查、13 项检查项(其中 5 项为关键项)、19 种奖励作弊和 26 种看似合理的错误(2026年11月10日)。关于公开演示
本演示证明了什么
已记录的检查项测试文件、数据库行和客户回复之间的一致性。预设的错误和奖励作弊探针测试特定的失败情形。它们并不能证明任务对前沿模型的难度,也不能证明可测量的训练收益。
验证器仍会漏掉一些以文字表述的矛盾,包括某些没有明确日期的交付承诺。这些情况需要人工审核。查看已知不足。
定价术语说明:已记录的 C12 检查项写的是“加价”(mark-up),但其公式是用成本除以 0.78,对应的是 22% 的目标毛利率,与流程规定一致。报告中保留了该检查项的原始文本。
从记录到任务
买方可评估的样本包。
从一个工作流开始,约定验收标准。每个范围的可用来源、使用权和专家审核都会逐一确认。
源记录
相关的邮件、聊天、流程文件和系统导出数据,附来源信息、修订记录和数据字典。覆盖范围取决于数据来源;并非每个工作流都具备所有记录。
任务与工作区
对于任务试点,需明确任务说明、初始状态、允许使用的工具和允许的修改。参考答案和评分逻辑放在智能体工作区之外。
完成的工作
定义所需的文件和系统变更。依据业务规则对最终状态评分,包括记录与面向客户的回复之间是否一致。
验收证据
约定针对正确的替代解法、看似合理的错误和绕过评分的尝试的检查项。提供带版本的结果、已知不足和重置说明。模型难度需要另行多次运行模型来衡量。
权利与审核
发布前确认允许用途、脱敏规则和交付条款。需要专业判断的地方,界定合格从业专家审核的范围,以及分歧的解决方式。
联络中心语音
真实的通话方式:一句话里两种语言。
我们正在界定英语、马来语和中文的客服、销售、核验和投诉通话的范围,包括通话中的语言切换。录音的可得性、语言变体、审核标签和数量均有待确认;以下规格为计划内容。
范围界定中
录音来源和同意要求见语音数据授权政策。
计划中的语音规格
| 组成部分 | 内容 |
|---|---|
| 音频 | 源格式的授权录音,或单独标注的重演通话;采样率和声道布局按来源约定 |
| 转写文本 | 逐字转写,标注说话人和时间戳,每个片段都带语言标签:en、ms、zh 或 mixed |
| 标签 | 质检项目、来电原因和处理结果(如来源中包含) |
| 去标识化 | 针对转写文本和音频、经界定范围的脱敏策略,附经审核的片段和处理记录。移除直接标识符并不能使声音变得匿名 |
| 元数据 | 经批准的字段,如呼叫方向、通话类型、时长、语言混合情况和说话人角色;不含客户直接标识符 |
| 质量检查 | 对约定样本进行独立审核;报告转写错误和说话人归属错误,并说明抽样方法和语码转换规则 |
| 交付 | 音频附 JSON 转写文本、数据说明表,每条录音附一份权利说明 |
{
"call": "demo-0001",
"direction": "inbound",
"type": "verification",
"segments": [
{ "speaker": "agent", "lang": "en",
"start": 0.0, "end": 3.2,
"text": "Thank you for calling. May I have your name, please?" },
{ "speaker": "customer", "lang": "mixed",
"start": 3.6, "end": 8.4,
"text": "Ya, nama saya [NAME]. I want to check kenapa kad saya kena block.",
"redactions": [{ "type": "person_name", "start": 4.5, "end": 5.3 }] },
{ "speaker": "agent", "lang": "ms",
"start": 8.8, "end": 11.5,
"text": "Baik, saya semak sekarang. Sila tunggu sebentar." }
]
}
你们是否与模型开发方竞争?
不。我们构建数据和验证器,不开发用于出售的模型;这些工作以我们团队在强化学习和后训练方面的研究为基础。模型及其训练平台始终归您所有。
目前已经准备好什么?
公开的工作流演示及其已记录的检查项现在即可查看。合作伙伴数据集和定制任务需要确认来源、权利和范围。语音数据正在界定范围。演示并不代表对应的客户数据集已可授权。
试点如何验收?
在工作开始前约定工作流、交付物、允许用途和审核标准。对于任务,评估参考输出、验证器结果和已知不足。如果难度很重要,则约定模型、工具访问权限、尝试次数和目标通过率区间,并在扩大批量之前完成测量。
把您的工作流交给我们。
告诉我们这项工作是什么、涉及哪些系统,以及正确结果必须包含什么。我们将界定记录、任务和验收证据的范围。