Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference
arXiv:2610.39562v1 Announce Type: cross Abstract: Multi-turn conversations with multimodal large language models (MLLMs) reuse input context, but long multimodal inputs increase time to first token (TTFT). Query-aware hard pruning reduces prefill tokens but permanently excludes information that later questions may require. We propose Raincheck, a framework for deferred multimodal prefill that decouples initial token selection from later information availability. An existing pruning method selec…