The Real Cost of Context: Predicting Compute Time and Energy Consumption in LLM Prefill and Decode Phases
arXiv:2610.39621v1 Announce Type: new Abstract: The integration of Large Language Models (LLMs) into agentic architectures has substantially increased token consumption, raising concerns about their energy footprint. Yet, estimating the energy consumption of proprietary API-based models remains challenging, and existing approaches focus primarily on generated output tokens while overlooking the impact of the large input contexts characteristic of agentic workflows. To address this gap, we syste…