Linear-Time Non-Local Codec Placement for Machine Learning Model Compression

arXiv:2610.14703v1 Announce Type: new Abstract: On-device inference imposes constraints on model storage and operand traffic and thus benefits from compressed operand representations, which mandate the use of encoding and decoding operators. However, na"{i}ve placement of codec operators can cancel the operand traffic gains sought from compression. In this work, I present an algorithm to place codec operators into a source computational graph to make it compatible with target hardware. The edges of the graph are colored so that each induced subgraph is weakly connected and has a maximal node that both postdominates all subgraph inputs and dominates all subgraph outputs; this unique node - dubbed the extit{absolute dominator} - is the natural location to observe the traffic flowing through the subgraph and learn parametric codec operators. The algorithm's time complexity is linear in the source graph's size. With respect to na"{i}ve local codec placement, the algorithm reduces modeled operand traffic by 24%-44% across four models, including 24% on SigLIP and 34% on SmolLM2.

Related

• BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression
• A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments
• Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference

Source: arXiv cs.LG | 2026-10-14