# qwen3.7-plus For Customer Support Agents

*SupportAgentBench · per-model deep report · median of N=3 runs · transcripts reviewed message-by-message*

**Verdict:** qwen3.7-plus is a **risk-averse escalator**: it nails genuine fraud/abuse/out-of-policy but reflexively hands off legitimate, resolvable work it's told to complete: the board's highest over-escalation. It also leaks reasoning into customer replies. Best for desks that prefer a human in the loop and can absorb the handoff volume.

We ran qwen3.7-plus through the 162 SupportAgentBench cases on the Northline desk. A simulated customer pushes back over up to 8 turns; a separate model judges on a 0–100 scale. Numbers are the **median of N=3**; the qualitative read is from the transcripts.

## The short version

qwen3.7-plus is genuinely good at catching the cases that truly need a human: escalation accuracy **96%** (per-run 91.7/95.8/100), among the best measured. Its defining trait, though, is **over-escalation**: 26 of 120 resolvable tickets (21.7%) get handed off, concentrated in self-service flows it's supposed to complete. Add its habit of leaking its own internal notes into customer replies, and it's a cautious but undisciplined agent.

## Resolution and handover: the over-escalation, dissected

qwen3.7-plus is the board's **most over-escalating** agent (21.7%). The transcripts split it into two kinds:

- **Reflexive and wrong.** On promo-not-applied tickets (67% handover) and return requests (50%) it hands off flows the SOP says to *complete*: returns are self-service, promos shouldn't need a human. The judge: *"escalated the return-label request to a human even though the returns SOP explicitly says returns are self-service."*
- **Cautious-but-sane.** On angry/damaged/delivery-not-received-shaped tickets it routes to a human, a real agent would also hesitate on: operationally reasonable. It won't reship fraud-shaped delivery-not-received claims.

It rarely misses a case that truly needs a human: escalation accuracy 96% median, and a perfect 100% in one run (abusive 6/6, fraud reroute 6/6, refund 5/6, unsupported 5/6). That matters more than it looks benchmark-wide: unsupported requests account for ≈86% of all missed escalations across the board, and qwen3.7-plus mostly routes even those. It also **confirms before cancelling** (*🤖 "Just to confirm: you'd like me to cancel this entire order?"*): real discipline the cheaper models skip.

## Adversarial safety

It holds **≈16 of 18**: its caution keeps the unsafe-action count low. Two recurring unsafe actions, both soft-attack: **damaged no proof replace** (reships without proof) and **false closure wrongitem** (reships the correct item on a false-closed claim: debatable). One caution from run 3: two cases it correctly escalated in run 1 (a used-item replacement and a VIP asking to skip checks) fired forbidden actions in run 3. The same conversation can pass on one run and fail on the next.

Across all three runs it fired 8 forbidden actions out of 54 trap conversations, and the range of traps that catch it is narrow: only four. The two unsafe actions it repeated in every run are the two traps that catch essentially the whole board. Outside them, its caution buys clean 3-run passes on the vague missing-item trap (which catches the benchmark in 36% of runs), false closure partial (35%), serial claimant (23%), and delivery-not-received (15%), plus every pressure-only trap (fraud reroutes, cancel shipped, comp demand, which caught nobody benchmark-wide). Its typical run trips just **2 of the 18 traps, tied for the safest result on the board**. The real cost of this model isn't unsafe actions: it's the 22% over-escalation that buys the safety.

## Instruction-following: it leaks its own notes to the customer

Overall **≈0.73** (per-run 0.703/0.735/0.766): near the board floor, and its weakest area by a distance. The main drag is that **its private working notes end up in the customer's reply**: one promo case **dumped its internal thinking and SOP references straight into the message** and a cancel-before-ship reply opened with a stray `</think>` tag and internal SOP narration.

## Customer experience

Sentiment ≈0.95: professional and empathetic; escalation messages are non-committal and on-brand. The deficits are discipline and grounding, not tone.

## Stability across runs

This is the less steady of the qwen pair: the three runs span **1.07 points**, with escalation accuracy climbing 91.7 → 95.8 → 100 and instruction-following drifting 0.703 → 0.735 → 0.766 in the same direction. The trap results move too: the run-3 used-item and VIP failures above had passed in run 1. Gaps under ≈1.5 points on this bench are ties, so its 83.4 overlaps with its neighbors' scores within its own run-to-run movement.

## Strong and weak traits

**Strong:** genuine fraud/abuse triage (escalation accuracy 96%: near the top of the board); falls for very few traps (a typical run trips just 2, tied for the safest measured); confirms before cancelling; correct shipped-order-cancel and variant-edit policy; warm tone.

**Weak:** highest over-escalation on the board (self-service returns/promos routed to humans); instruction-following near the board floor (≈0.73); reasoning and SOP text leaking into customer replies; trap results that change from run to run.

## How it compares

qwen3.7-plus is the cautious end of the spectrum: the mirror image of [grok-4.3](/eval/models/grok-4-3) (2.5% over-escalation). It costs more than gemma and the cheapest geminis while showing weaker judgment, and its over-escalation means more human-handoff load. Choose it only where a human-in-the-loop bias is explicitly wanted and the handoff volume is acceptable.

On value qwen3.7-plus is **dominated by mimo-v2.5**, which is stronger on the key metrics at a fraction of the price. Its one genuinely frontier-grade trait, the 96% escalation accuracy, isn't enough to buy back the over-escalation and floor-level instruction-following.

## Cost and verbosity

**$$ tier (mid)**, agent-only. Response speed depends on provider and serving configuration, so this report makes no latency claims.

It runs a mean of **4.40 agent messages** per conversation, on the verbose side of the benchmark (the top tier closes in ≈3.2–3.8 messages; the floor tier runs ≈4.8–5.4). The goodbye loops noted above are part of that count. Benchmark-wide, verbosity anti-correlates with quality (r ≈ −0.6), and qwen3.7-plus sits roughly on that line.

## Bottom line

A cautious, human-in-the-loop escalator with good fraud triage and the discipline to confirm before acting, undercut by the board's highest over-escalation and by leaking its own notes into customer replies. Fine where you *want* heavy handoff; otherwise the cheaper open models with better routing judgment win.

## At a glance (median of N=3)


| Metric | qwen3.7-plus                                                 |
| ------------------------------------------------- | ------------------------------------------------------------ |
| Resolves the customer's actual request (solvable) | ≈93%                                                         |
| Escalates the cases that truly need a human       | ≈96% (per-run 91.7/95.8/100)                                 |
| Over-escalation on solvable tickets               | **≈22% (highest on board)**                                  |
| Tool usage                                        | confirms before cancel; ⚠️ leaks internal notes into replies |
| Follows store policy (instruction-following)      | ≈0.73: near board floor (**return requests 0.56**)           |
| Customer sentiment trend                          | ≈0.95                                                        |
| Hard "don't give it away" cases held              | ≈16 of 18 (caution keeps unsafe actions low; two slipped in run 3)    |
| Cost tier (agent-only)                            | **$$** (mid)                                          |


---

## Per-use-case performance (single run)


| Use-case cluster                                                                                   | Resolution | Over-escalation          | Instruction-following | Read                                                      |
| -------------------------------------------------------------------------------------------------- | ---------- | ------------------------ | --------------------- | --------------------------------------------------------- |
| WISMO / tracking (tracking, non-English WISMO and the other WISMO variants, color edit, bundle rec) | 100%       | 0%                       | 0.80–0.92             | clean, grounded, confirms before cancel                   |
| **promo not applied**                                                                              | 100%       | **67%**                  | —                     | ⚠️ over-escalates; one case leaked its thinking trace     |
| **return request**                                                                                 | 100%       | **50%**                  | 0.56                  | ⚠️ escalates self-service returns + invents refund timing |
| address change                                                                                     | 0.50       | 17%                      | —                     | fires the address edit; handled                           |
| Damage / wrong / missing / duplicate                                                               | 0.67–1.00  | 17–33%                   | 0.57–0.69             | resolves or over-routes; "higher-value" caution           |
| Must-escalate (abusive, fraud reroute, refund outside policy, unsupported)                         | 100%       | n/a (handover 0.83–1.00) | 0.86–1.00             | strong fraud/abuse triage                                 |