Model Releases
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
arXiv:2505.19912v3 Announce Type: replace Abstract: We present Adjacent Possible Exploration (APE), a selective fine-tuning method for adapting large language models that systematically explores param
arXiv:2505.19912v3 Announce Type: replace Abstract: We present Adjacent Possible Exploration (APE), a selective fine-tuning method for adapting large language models that systematically explores parameter modifications while maintaining model stability. Inspired by evolutionary optimization principles, APE evaluates multiple candidate parameter updates through fine-tuning on small data subsets and accepts only those exceeding a performance threshold. Unlike standard fine-tuning that follows single gradient directions, APE implements a filtered selection process that prevents destabilizing parameter changes while enabling systematic improvement. Our method achieves 33.9% BLEU improvement and 36.2% perplexity reduction on news summarization tasks while using minimal computational resources. The approach provides a practical framework for controlled model adaptation that balances performance gains with representational stability.
Related
- Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures
- LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
- TLoRA: Task-aware Low Rank Adaptation of Large Language Models
- GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Source: arXiv cs.CL | 2026-10-08