Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search

arXiv:2608.00078v1 Announce Type: new Abstract: Deploying convolutional neural networks generated by large language models (LLMs) on real mobile hardware requires more than GPU validation accuracy: INT8 TensorFlow Lite export, delegate selection, and on-device latency jointly determine whether a model is usable. We present an automated mobile deployment pipeline that closes the loop from QLoRA fine-tuning of an architecture-generating LLM through GPU evaluation, INT8 export, and physical-device benchmarking to gated augmentation of the training corpus. The pipeline is fully scripted and runs cycle-by-cycle without manual intervention, with resume support after interruptions. We evaluate the same frozen protocol on two benchmarks, CIFAR-10 and CIFAR-100, on a Samsung SM-P613 tablet (seed 42, 20 models per cycle, cycles 0-6). On CIFAR-10, cycle 1 is gate-accepted and improves the mobile deployment score approximately 25.6x over the baseline with a mean quantized accuracy of 46.9%; later cycles raise GPU accuracy but fail the non-decreasing mobile gate. On CIFAR-100, the pre-QLoRA baseline retains the best mobile score; iterative rounds improve GPU accuracy (up to 26.2%) yet cannot surpass cycle 0 on-device, and the training pool stalls at 19 examples after the first accepted round. Together, the two studies show that closed-loop GPU fine-tuning does not guarantee monotonic mobile gains, especially on harder classification tasks, and that multi-dataset, on-device measurement is needed to stress-test deployment objectives. We release per-cycle metrics with 95% confidence intervals, all figures, and complete reproduction commands.

Related

• UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
• Toward a Modular Architecture for Embedded AI Agent Systems at the Edge
• Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization

Source: arXiv cs.CV | 2026-08-04