Model Releases
SLIM: Simplex-Lattice Interpolation Merging
arXiv:2610.01037v1 Announce Type: new Abstract: Optimizing merging coefficients for large language models can require many costly benchmark evaluations. We propose extbf{Simplex-Lattice Interpolation
arXiv:2610.01037v1 Announce Type: new Abstract: Optimizing merging coefficients for large language models can require many costly benchmark evaluations. We propose extbf{Simplex-Lattice Interpolation Merging (SLIM)}, which constructs a quadratic surrogate of aggregate performance on the coefficient simplex using a classical mixture design. Evaluations of individual experts and equal-weight pairs determine the surrogate with the minimum number of measurements needed to identify a general quadratic on this domain. SLIM then optimizes the surrogate without further target-metric evaluations. Experiments on two model architectures demonstrate accurate prediction of unseen multi-expert mixtures and competitive merge performance under limited evaluation budgets. Matched-budget comparisons show that structured evaluation points improve prediction fidelity over random designs, including those using regularized fitting.
Related
- ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
- Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
- Merge-Bench: Resolve Merge Conflicts with Large Language Models
Source: arXiv cs.LG | 2026-10-02