Static trace report

Inference Autopsy Report

Source
examples\traces\fake.jsonl
Generated
2026-07-05T04:07:06+00:00
Profiles
code-completion, mixed-realistic, rag-long, short-chat
Models
fake-model
Load Modes
closed_loop

Executive Summary

Top-level health signals derived from the saved trace file.

Requests 50 all trace records
Success Rate 86.0% successful requests
Error Rate 14.0% non-success requests
Latency p95 10687 ms request latency
TTFT p95 1000 ms time to first token
ITL p95 117 ms inter-token latency
Tail Ratio 4.51x p99 / p50 latency
Stalls 7 stream stall count

Diagnosis

Evidence-backed labels. These describe black-box symptoms, not definitive backend root causes.

Context Golem

high severity medium confidence

Evidence

  • rag-long TTFT p95 is 1018.0 ms.
  • short-chat TTFT p95 is 236.0 ms.
  • rag-long is 4.31x slower to first token.

Likely external symptom

Longer prompt-shaped workloads are associated with first-token delay.

Possible internal causes

  • prompt prefill cost
  • long-context processing overhead
  • missing or ineffective prefix reuse

Suggested next tests

  • Bucket TTFT by input token count.
  • Run repeated-prefix cache-sensitive workload.
  • Compare with shorter context windows.

Tail Wizard

medium severity high confidence

Evidence

  • Request latency p50 is 2566.0 ms.
  • Request latency p99 is 11570.0 ms.
  • Tail ratio p99/p50 is 4.51.

Likely external symptom

A small share of requests are much slower than typical requests.

Possible internal causes

  • queueing variance
  • rare long prompts or outputs
  • provider-side retries or intermittent stalls

Suggested next tests

  • Inspect worst requests by latency.
  • Group tail latency by profile and concurrency.
  • Repeat with more requests to verify tail stability.

Stream Wall Breaker

medium severity high confidence

Evidence

  • Total stream stall count is 7.

Likely external symptom

Generated tokens arrive with large visible gaps.

Possible internal causes

  • stream delivery stalls
  • backend scheduling interruptions
  • network or provider buffering variance

Suggested next tests

  • Inspect token gap histogram.
  • Compare streaming and non-streaming paths.
  • Check worst requests by ITL.

Overall Metrics

Canonical percentiles reused from the metrics layer.

MetricSamplesp50p90p95p99
TTFB 48 93 ms 102 ms 104 ms 105 ms
TTFT 45 213 ms 954 ms 1000 ms 1018 ms
Request Latency 50 2566 ms 10418 ms 10687 ms 11570 ms
ITL Mean 45 44 ms 102 ms 117 ms 123 ms
ITL p95 45 41 ms 102 ms 117 ms 123 ms
Output TPS 45 22.7 tok/s 29.2 tok/s 30.1 tok/s 30.1 tok/s

Charts

Lightweight static charts for trend reading. Exact values remain in the tables.

TTFT p95 by Concurrency

c1
1000 ms
c2
995 ms
c3
919 ms
c4
1002 ms

Request Latency p95 by Concurrency

c1
10518 ms
c2
11390 ms
c3
8991 ms
c4
4176 ms

Error Rate by Concurrency

c1
7.7%
c2
23.1%
c3
16.7%
c4
8.3%

TTFT by Cache Mode

n/a

Request Latency Histogram

<1s
5
1-2s
15
2-5s
21
5-10s
1
>=10s
8

Profile Breakdown

Grouped summaries expose workload-specific and load-sensitive behavior.

GroupRequestsSuccessError TTFT p95Latency p95ITL p95Tail Ratio
code-completion 7 100.0% 0.0% 230 ms 11570 ms 123 ms 1.10x
mixed-realistic 7 100.0% 0.0% 230 ms 4176 ms 48 ms 1.18x
rag-long 10 100.0% 0.0% 1018 ms 2785 ms 45 ms 1.42x
short-chat 26 73.1% 26.9% 236 ms 10000 ms 46 ms 5.44x

Concurrency Breakdown

Grouped summaries expose workload-specific and load-sensitive behavior.

GroupRequestsSuccessError TTFT p95Latency p95ITL p95Tail Ratio
1 13 92.3% 7.7% 1000 ms 10518 ms 100 ms 5.12x
2 13 76.9% 23.1% 995 ms 11390 ms 117 ms 4.15x
3 12 83.3% 16.7% 919 ms 8991 ms 123 ms 4.96x
4 12 91.7% 8.3% 1002 ms 4176 ms 110 ms 3.79x

Profile + Concurrency Breakdown

Grouped summaries expose workload-specific and load-sensitive behavior.

GroupRequestsSuccessError TTFT p95Latency p95ITL p95Tail Ratio
code-completion@c1 2 100.0% 0.0% 214 ms 10525 ms 118 ms 1.00x
code-completion@c2 2 100.0% 0.0% 230 ms 11570 ms 117 ms 1.02x
code-completion@c3 2 100.0% 0.0% 213 ms 10418 ms 123 ms 1.16x
code-completion@c4 1 100.0% 0.0% 192 ms 10687 ms 110 ms 1.00x
mixed-realistic@c1 1 100.0% 0.0% 224 ms 3817 ms 38 ms 1.00x
mixed-realistic@c2 2 100.0% 0.0% 220 ms 3548 ms 39 ms 1.22x
mixed-realistic@c3 2 100.0% 0.0% 230 ms 3710 ms 47 ms 1.37x
mixed-realistic@c4 2 100.0% 0.0% 217 ms 4176 ms 48 ms 1.24x
rag-long@c1 4 100.0% 0.0% 1018 ms 2057 ms 44 ms 1.05x
rag-long@c2 2 100.0% 0.0% 995 ms 2785 ms 38 ms 1.07x
rag-long@c3 2 100.0% 0.0% 919 ms 1950 ms 45 ms 1.05x
rag-long@c4 2 100.0% 0.0% 1002 ms 2598 ms 34 ms 1.01x
short-chat@c1 6 83.3% 16.7% 211 ms 10000 ms 44 ms 5.13x
short-chat@c2 7 57.1% 42.9% 223 ms 10000 ms 46 ms 7.23x
short-chat@c3 6 66.7% 33.3% 242 ms 2288 ms 40 ms 1.42x
short-chat@c4 7 85.7% 14.3% 236 ms 3186 ms 47 ms 1.94x

Cache Summary

Black-box cache-sensitive comparison. The report does not claim definitive cache hits.

SignalValue
Cold TTFT p95n/a
Warm TTFT p95n/a
Repeated-prefix TTFT p95n/a
Repeated-exact TTFT p95n/a
Cold / warm ration/a
Cold / repeated-prefix ration/a

Worst Requests by Latency

Individual traces that explain the aggregate tail.

RequestProfileConcurrencyStatus TTFTLatencyOutput TokensError
req_00005 code-completion 2 success 203 ms 11570 ms 112 n/a
req_00045 code-completion 2 success 230 ms 11390 ms 96 n/a
req_00015 code-completion 4 success 192 ms 10687 ms 96 n/a
req_00020 code-completion 1 success 214 ms 10525 ms 88 n/a
req_00040 code-completion 1 success 173 ms 10518 ms 104 n/a
req_00010 code-completion 3 success 176 ms 10418 ms 104 n/a
req_00000 short-chat 1 timeout n/a 10000 ms 0 timeout: Request timed out before first byte.
req_00025 short-chat 2 timeout n/a 10000 ms 0 timeout: Request timed out before first byte.
req_00030 code-completion 3 success 213 ms 8991 ms 72 n/a
req_00035 mixed-realistic 4 success 200 ms 4176 ms 80 n/a

Worst Requests by TTFT

Individual traces that explain the aggregate tail.

RequestProfileConcurrencyStatus TTFTLatencyOutput TokensError
req_00024 rag-long 1 success 1018 ms 1845 ms 24 n/a
req_00003 rag-long 4 success 1002 ms 2598 ms 48 n/a
req_00036 rag-long 1 success 1000 ms 2057 ms 24 n/a
req_00009 rag-long 2 success 995 ms 2591 ms 48 n/a
req_00033 rag-long 2 success 954 ms 2785 ms 48 n/a
req_00027 rag-long 4 success 923 ms 2566 ms 48 n/a
req_00018 rag-long 3 success 919 ms 1861 ms 24 n/a
req_00012 rag-long 1 success 911 ms 1807 ms 24 n/a
req_00048 rag-long 1 success 901 ms 1958 ms 24 n/a
req_00006 rag-long 3 success 870 ms 1950 ms 24 n/a

Methodology

How to read this artifact.

This report was generated from saved JSONL traces. Percentiles, reliability rates, grouped summaries, cache-sensitive signals, and diagnoses reuse the same project modules as the CLI.

Missing values are shown as n/a. Diagnosis text describes externally visible symptoms and uses cautious black-box language.

Record count: 50.