Benchmarks
Measured, or not shown.
This page is the frame for DynaLM’s public performance results. Every number will arrive with the conditions that produced it. Until a reproducible result set is published, the metrics below stay empty — never zero, never estimated.
Reproducible performance results are being prepared.
Every number on this page will come with its hardware, model, quantization, backend, method and reproduction commands. Until then, nothing is shown.
Metrics
What will be reported.
Each metric has a unit, a direction and a definition. Values shown are from the primary DynaLM run.
Decode throughput
—tok/s: not yet measured
Output tokens generated per second for one stream, after the first token.
↑ higher is betterAwaiting run
Prefill throughput
—tok/s: not yet measured
Prompt tokens processed per second before generation starts.
↑ higher is betterAwaiting run
Time to first token
—ms: not yet measured
Median delay from request arrival to the first streamed token.
↓ lower is betterAwaiting run
Multi-user throughput
—tok/s: not yet measured
Total output tokens per second across all concurrent requests.
↑ higher is betterAwaiting run
Inter-token latency P50
—ms: not yet measured
Median gap between consecutive tokens within a stream.
↓ lower is betterAwaiting run
Inter-token latency P95
—ms: not yet measured
95th-percentile gap between consecutive tokens.
↓ lower is betterAwaiting run
Inter-token latency P99
—ms: not yet measured
99th-percentile gap between consecutive tokens — the stutter users notice.
↓ lower is betterAwaiting run
Memory usage
—MB: not yet measured
Peak resident memory of the server process, model loaded.
↓ lower is betterAwaiting run
Failed requests and timeouts
—count: not yet measured
Requests that errored or exceeded the timeout during the run.
↓ lower is betterAwaiting run
Methodology
How results are produced.
- Same inputsEvery engine loads the same model file at the same quantization.
- Comparable settingsContext length, batch limits, threads and sampling are matched and listed.
- Warm-up excludedResults are recorded after warm-up runs; load time is reported separately.
- Repeated runsEach scenario is repeated; the median is reported with its spread.
- Full disclosureHardware, OS, power state, versions and flags are published with each result.
- Raw dataRaw outputs and the commands that produced them are published for reproduction.
Configuration
Hardware and software.
Model selection, quantization and backend are fixed per result set.
- Hardware
- Not yet publishedCPU / GPU model, memory, power state
- Operating system
- Not yet publishedOS name and version
- Model
- Not yet publishedModel name and parameter count
- Quantization
- Not yet publishede.g. Q4_K_M — same file for every engine
- Backend
- Not yet publishedCPU (ISA), Metal, Vulkan or CUDA
- DynaLM version
- Not yet publishedRelease tag or commit
- Comparison baseline
- Not yet publishedEngine, version and flags
- Load generator
- Not yet publishedTool, prompt set, output length
Results
Single-user and multi-user.
Single user
One request at a time. Measures raw decode speed and latency without contention.
| Engine | Concurrency | Decode throughput tok/s | Prefill throughput tok/s | Memory usage MB |
|---|---|---|---|---|
No published runs for this scenario. Cells will fill from benchmarks.json. | ||||
Decode throughput tok/s
Prefill throughput tok/s
Memory usage MB
Multi-user
Several concurrent requests. Measures how scheduling and batching hold up under load.
| Engine | Concurrency | Time to first token ms | Multi-user throughput tok/s | Inter-token latency P50 ms | Inter-token latency P95 ms | Inter-token latency P99 ms | Failed requests and timeouts count |
|---|---|---|---|---|---|---|---|
No published runs for this scenario. Cells will fill from benchmarks.json. | |||||||
Time to first token ms
Multi-user throughput tok/s
Inter-token latency P50 ms
Inter-token latency P95 ms
Inter-token latency P99 ms
Failed requests and timeouts count
Reliability
Failed requests and timeouts.
Throughput means little if requests fail. Each multi-user run reports the number of requests that errored or exceeded the timeout, alongside the timeout used. A run with failures is published with them, not discarded.
Reproduce
Run it yourself.
The CLI includes a benchmark command for measuring your own machine (from the project README). It is not a substitute for the published method.
# Measure your own machine
dynalm benchmark qwen2.5:1.5bReproduction commands will be published with the first result set.
Exact commands, prompt sets and raw outputs will be published with the first result set.
Limitations
Read before comparing.
- Results apply only to the listed hardware, model, quantization and backend.
- Comparisons use the same model file and comparable settings for every engine.
- Where a baseline engine is faster, the result is shown as measured.
Earlier measurements, with their own stated setup and limits, are documented in the repository: docs/benchmarks.md (opens in a new tab)
Machine-readable data for this page: benchmarks.json