v0.40.17-rc1: mlxrunner: batch speculative verification (#19663)

Open source GitHub · Ollama Source language: English

mlxrunner: batch speculative verification Keep drafting active for concurrent generation by sharing target verification while preserving per-request sampling and cache rollback.

Excerpt

mlxrunner: batch speculative verification Keep drafting active for concurrent generation by sharing target verification while preserving per-request sampling and cache rollback.

Short excerpt from the source — open the original for the full text.

Sources

← Back to the daily brief