Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging
Pengyang Zhou, Xiaobin Tu, Zhengxi Liu, Rongkun Xue, Haochen Li, Miancan Liu, Ziyuan Chen, Yinggui Wang, et al.
cs.AI
Quick verdict
TAC-Merge traces cross-layer multimodal influence with curvature graphs and tunes merge weights via trust-region control; solid gains, but no code release.
Summary
TAC-Merge is a method paper on merging LoRA-adapted multimodal experts into one shared model. Motivated by cross-layer propagation of expert updates, it couples two modules: Multimodal Influence Mapping (MIM) builds bipartite influence graphs from expert-region sources to hidden readouts at various depths and token positions, measures readout sensitivity to coefficients, and defines a fusion objective combining Ollivier-Ricci curvature matching with KL agreement to expert next-token predictions; Coupled Merge Control (CMC) fits a local quadratic trust-region model over a compact SVD-derived coefficient subspace and accepts a proposal only if it measurably improves the objective. On Qwen3.5-4B and Qwen3-VL-8B with eight seen and four unseen tasks, TAC-Merge achieves the best merging-method averages (seen 78.50/77.50, unseen 68.10/66.50), exceeding the strongest baselines by 3.43/2.34 (4B) and 2.42/1.94 (8B) points. Ablations show MIM matters more than CMC (removing MIM costs 2.38 seen points on 4B vs. 1.47 for CMC), while modality-contrast channels and directional coupling each contribute under 1.1 points. The paper reports no significance testing, no computational cost comparison against baselines, and promises no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Layer-wise merging cannot capture how expert updates influence downstream representations and image-instruction interactions, motivating cross-layer, modality-aware coefficient selection (Sec. 1, CH1/CH2).
- Combining Ricci-curvature matching on multimodal influence graphs with expert-prediction KL (Eq. 7) provides a fusion objective that better preserves expert capabilities than parameter-space or activation-based objectives.
- Joint trust-region optimization over a compact coefficient subspace with interaction modeling (CMC) outperforms direct candidate search under the same evaluation budget (Table 3, w/o CMC: -1.47 seen on 4B).
- TAC-Merge achieves the best average among nine compared merging methods on both seen and unseen tasks across Qwen3.5-4B (78.50/68.10) and Qwen3-VL-8B (77.50/66.50) (Tables 1-2).
- Gains on unseen tasks (e.g., +2.34 on 4B over the strongest baseline) show the merged coefficients generalize beyond the expert training distributions.
Strengths
Clear, well-motivated problem framing beyond layer-wise merging
Sec. 1 formalizes CH1/CH2: expert updates propagate across depth and couple visual-language computation, which layer-wise methods (TA, TIES, RobustMerge) ignore. Tables 1-2 support this: RegMean++ (cross-layer) beats TIES by 6.66 seen points on 4B but trails TA by 1.00 on unseen, motivating joint cross-layer + modality treatment.
Technically coherent two-module design with principled optimization
MIM (Sec. 3.3) grounds influence in measurable readout sensitivities (Eq. 4); CMC (Sec. 3.4) uses SVD stable-rank subspace selection, individual+paired variations to fit interactions (Eq. 10), exact spectral trust-region solving (Eq. 12, Moré & Sorensen 1983), and a non-wasteful acceptance check against the uniform merge.
Consistent gains on both backbones and both task groups
Ablations attribute value to each module with matched budgets
Honest treatment of the transfer-vs-calibration trade-off
Weaknesses
No statistical significance testing anywhere
No computational cost analysis for a notably expensive method
Key implementation details absent, blocking reproduction
The geometric objective is asserted, not validated
Substantial residual gaps to individual experts
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TAC-Merge (arXiv 2609.12897, Alibaba Cloud) targets multimodal model merging for MLLMs: consolidating K LoRA task experts into one static shared model. The paper's diagnosis (Sec. 1) is that layer-wise merging — including recent multimodal variants (RobustMerge, OptMerge) — ignores two facts: expert updates propagate across depth (CH1), and visual/language updates couple through shared representations, so their weights must be chosen jointly (CH2). The claimed contribution is a two-module remedy: Multimodal Influence Mapping (MIM) and Coupled Merge Control (CMC), evaluated on two Qwen backbones over eight seen and four unseen tasks. The headline claim — best average among nine merging methods on both task groups on both backbones — is supported by Tables 1-2 as reported, but statistical support for the margins is thin (see below).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TAC-Merge and what problem does it solve?
TAC-Merge is a model merging method for multimodal large language models. It combines several task-specialized LoRA experts (e.g., trained for VQA, OCR, captioning, object localization) into a single deployable model. Unlike standard layer-wise merging, it traces how each expert's update propagates across network layers and across the image/instruction pathways, then jointly selects per-region merge weights to preserve those effects.
How much does TAC-Merge improve over existing merging methods?
On Qwen3.5-4B it reaches an average of 78.50 on eight seen tasks and 68.10 on four unseen tasks, beating the strongest of nine compared merging baselines by 3.43 and 2.34 points respectively; on Qwen3-VL-8B the averages are 77.50 and 66.50, with gains of 2.42 and 1.94 points. The largest single margin is ImageNet on the 4B model: 88.08 vs 48.18 for RobustMerge.
What are the two modules of TAC-Merge?
Multimodal Influence Mapping (MIM) builds influence graphs that link each expert-region update to hidden readouts at different depths and token positions, and defines a merging objective from Ollivier-Ricci curvature matching plus KL agreement with expert predictions. Coupled Merge Control (CMC) fits a local quadratic model of how coefficient adjustments interact, within a compact SVD-derived subspace, and picks the final regional weights via trust-region optimization, keeping the uniform merge if no measured improvement is found.
Is code available for TAC-Merge?
No. The paper does not mention a code release, and several implementation details needed for reproduction (calibration set size, region definitions, control dimension, readout positions) are not specified. As of this paper's text, reproduction requires re-implementing the full pipeline from scratch.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text ends mid-way through the References section (last entry: Zhang et al. 2025, cross-modal information flow); any references alphabetically after that entry would be absent, though all in-text citations visible in the body were resolvable within the provided text.
- Figure 2's sensitivity curves are described in the caption but the actual plotted values cannot be read from text extraction; sensitivity claims rely on the prose description only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.