Model Releases
Measurement-Driven Sub-Network Selection for On-Premise Retrieval-Augmented Factory Agents
arXiv:2609.02760v1 Announce Type: new Abstract: On-premise assistants can give factory workers conversational access to machine documentation, but models capable of the task rarely fit shop-floor hard
arXiv:2609.02760v1 Announce Type: new Abstract: On-premise assistants can give factory workers conversational access to machine documentation, but models capable of the task rarely fit shop-floor hardware. We show that after structural compression and retrieval-grounded adaptation, model size is no longer a reliable predictor of adapted answer quality: general capability falls almost linearly with parameter count, while judged retrieval-augmented answer quality does not. We therefore treat deployment as a post-adaptation selection problem, committing one sub-network per device on judged answer quality and measured on-device throughput under a configurable general-capability floor and memory budget; rules that optimize size, speed, or quality alone each give up capability or throughput. A weight-shared supernetwork trained with sandwich-style in-place distillation keeps this selection inexpensive. In a manufacturing-manual case study, extraction costs 13.7 percent of the unpruned model's judged quality and retrieval-grounded distillation returns it to within 4.6 percent, recovering two thirds of the loss, and the same assistant runs across three heterogeneous edge tiers at 1.3 to 5 watts standby.
Related
- LARAG: Link-Aware Retrieval Strategy for RAG Systems in Hyperlinked Technical Documentation
- Retrieval-Augmented LLM Agents: Learning to Learn from Experience
- CausalRAG2: Hierarchical Causal Knowledge Graph Design for RAG
- LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking
Source: arXiv cs.AI | 2026-09-03