Variational Low-Rank Adaptation for Efficient CNN Fine-Tuning

Abstract

We propose LoR-VC (Low-Rank Variational Convolutional adaptation) to close this gap. LoR-VC first reshapes the pre-trained convolutional kernel from its native tensor form $(C_{out}, C_{in}, k, k)$ into a 2D matrix of dimension $(kC_{out}, kC_{in})$. It then applies truncated SVD to this reshaped matrix, chosen for its stable convergence properties, to construct a fixed spectral basis from the frozen pre-trained weights. Within this decomposition, LoR-VC introduces a compact variational core $\Theta$ of low rank $(r, r)$, which is the only trainable component and is responsible for capturing the representational shift induced by new data.