In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

Summary

This paper proposes In-Context Guidance Multitask Optimization (ICG-MTO), a framework that uses a frozen numerical foundation model (TabPFN v2.5) to improve estimation of inter-task coupling in multitask optimization when evaluation budgets are small. Evaluated solutions are reorganized into algorithm-specific in-context queries: for multitask Bayesian optimization (ICG-MTBO), each task's solutions are ranked into better/worse fitness classes, and TabPFN performs directional cross-task classification whose cross-entropy yields a bounded guidance matrix; for MFEA-II (ICG-MFEA-II), decision-space overlap between elite sets plays the analogous role. The guidance enters as a Gaussian prior over the coupling parameters in a MAP objective whose weight lambda(t)=e^(-0.05t) anneals to zero, so the optimizer falls back to standard likelihood-based estimation as data accumulate. On nine two-task synthetic benchmarks (30-D and 50-D, 100 evaluations per task, 20 runs), ICG-MTBO attains the best average rank at every budget (1.89 to 1.56 and 1.67 to 1.39), and against its unguided counterpart MTBO-LCB it wins 101, ties 37, and loses 6 of 144 comparisons, with gains concentrated early. A robot-arm reaching study and a LogEI variant support generality. Main limitations: the justification is heuristic (pseudo-observation analogy), no ablation separates informed guidance from generic early regularization, the MFEA-II study initializes from 200 evaluated samples per task, and no code release is stated.