---
rank: 1
kind: ranking
id: rk_3c47eeb7b47a16bd
title: Models for summarizing support tickets
subject: model
question: Which model should summarize support tickets?
status: draft
as_of: 2026-10-24T18:00:00Z
fresh_for: 3d
method: weighted_sum
normalize: minmax
ties: competition
missing: worst
decide: min_lead=0.05 min_confidence=0.7 on_close=explore top_k=3
snapshot: rs_fd33801b2160ba77
---

# Models for summarizing support tickets

Which model should summarize support tickets?

## Criteria

| id | label | weight | better | unit | normalize | half_life | max_age | must | how |
|---|---|---|---|---|---|---|---|---|---|
| eval_quality | Quality on your evals | 0.35 | higher | ratio | none | 14d | 60d | | Pass rate on your own eval set for this task. |
| cost_per_mtok | Cost per million tokens | 0.2 | lower | usd | log | 14d | 90d | | Blended cost per million tokens, from your own invoices or usage records. |
| ttft_ms | Time to first token | 0.1 | lower | ms | log | 3d | 14d | | Median time to first token, measured on your own calls. |
| throughput | Throughput | 0.1 | higher | tokens_per_s | minmax | 3d | 14d | | Output tokens per second, measured on your own calls. |
| tool_call_success | Tool-call success | 0.15 | higher | ratio | none | 7d | 30d | | Share of tool calls that were well formed and worked, from your run logs. |
| refusals | Refusals on your tasks | 0.1 | lower | ratio | none | 7d | 30d | | Share of your legitimate tasks the model declined, from your own logs. |

## Items

| rank | id | label | score | low | high | move | uri | group | eval_quality | cost_per_mtok | ttft_ms | throughput | tool_call_success | refusals |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | it_90f4e2e77e8d82e4 | Small fast model | 0.9315 | 0.917811 | 0.946179 | | | | 0.84 ±0.025 n=400 | 0.6 | 180 | 210 | 0.93 | 0.02 |
| 2 | it_dfef73cf031c3326 | Mid-size model | 0.768376 | 0.756293 | 0.780574 | | | | 0.88 ±0.022 n=400 | 2.5 | 320 | 120 | 0.95 | 0.01 |
| 3 | it_0d180dcf9e2b24ea | Large general model | 0.589916 | 0.578228 | 0.60069 | | | | 0.91 ±0.02 n=400 | 9 | 650 | 70 | 0.97 | 0.01 |
| 4 | it_0c8c577f4c752cb4 | Long-context model | 0.581426 | 0.567123 | 0.595366 | | | | 0.87 ±0.024 n=400 | 6 | 900 | 60 | 0.94 | 0.03 |

## Sources

| id | kind | trust | label | uri |
|---|---|---|---|---|
| own_evals | own | 1 | Your 400-ticket eval set | |
| usage | own | 1 | Your usage records | |

## Notes

Example data. Replace every value with your own.

When to use: Rank models for one task on your evals, with standard errors, intervals and each model's chance of being best.