Local Ai
LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service
arXiv:2608.13144v1 Announce Type: cross Abstract: As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sa
arXiv:2608.13144v1 Announce Type: cross Abstract: As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has become a central concern. Existing semantic caching methods largely rely on empirical similarity thresholds; while such thresholds improve hit rates, they tend to introduce silent misclassifications near decision boundaries. To address this, we propose exttt{LipCache}, a certified semantic caching framework for image classification. Without modifying the existing deployed main model, exttt{MainNet}, the framework introduces a lightweight network, exttt{GuardNet}, that maps inputs into a low-dimensional feature space subject to a Lipschitz constraint. It then computes a per-sample certified reuse radius from the local classification margin and the spectral norm of the classification head. At runtime, a cached result is reused only when the query feature falls inside the certified reuse ball; otherwise, the query falls back to exttt{MainNet}. Thus, cache hits are transformed from empirical threshold tests into geometric certification decisions with explicit theoretical boundaries. Across standard image classification tasks like CIFAR, Tiny-ImageNet, and SVHN, exttt{LipCache} achieves a measured speedup of up to 1.65imes with limited end-to-end accuracy degradation, while all accepted cache hits satisfy the exttt{GuardNet}-side certified-consistency condition. Furthermore, an enhanced exttt{GuardNet} training recipe substantially improves cache hit rates in the Tiny-ImageNet multi-class extension while maintaining a certified-consistency rate of 100%. These results demonstrate that per-sample certified reuse can reduce main-model fallback while preserving theoretical consistency, providing a feasible approach to reliable cache-assisted inference at the edge.
Related
- Collaborative Edge-to-Server Inference for Vision-Language Models
- MedFormer-UR: Uncertainty-Routed Transformer for Medical Image Classification
- Dot-Flik: A Scalable Edge AI Architecture for Distributed Insect Monitoring
Source: arXiv cs.AI | 2026-08-14