Model Releases
An Empirical Study of On-Device Translation for Real-Time Live-Stream Chat on Mobile Devices
arXiv:2601.02641v2 Announce Type: replace Abstract: Despite its efficiency, there has been little research on the practical aspects required for real-world deployment of on-device AI models, such as t
arXiv:2601.02641v2 Announce Type: replace Abstract: Despite its efficiency, there has been little research on the practical aspects required for real-world deployment of on-device AI models, such as the device's CPU utilization and thermal conditions. In this paper, through extensive experiments, we investigate two key issues that must be addressed to deploy on-device models in real-world services: (i) the selection of on-device models and the resource consumption of each model, and (ii) the capability and potential of on-device models for domain adaptation. To this end, we focus on a task of translating live-stream chat messages and manually construct LiveChatBench, a benchmark consisting of 1,000 Korean-English parallel sentence pairs. Experiments on five mobile devices demonstrate that, although serving a large and heterogeneous user base requires careful consideration of highly constrained deployment settings and model selection, the proposed approach nevertheless achieves performance comparable to commercial models such as GPT-5.1 on the well-targeted task. We expect that our findings will provide meaningful insights to the on-device AI community.
Related
- Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
- Revisiting Training Scale: An Empirical Study of Token Count, Power Consumption, and Parameter Efficiency
- Real-Time Deadlines Reveal Fragile Temporal Adaptation in LLM Strategic Dialogues
Source: arXiv cs.AI | 2026-10-01