LyndHub
Jadwal Sholat

Memuat jadwal sholat…

← All journalsSearch index

Ilmu Komputer & AI editorial

Open AccessOA2026

In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

ICG-MTO: A three-stage framework that uses frozen numerical foundational models to regularize inter-task coupling estimation under scarce evaluation budgets
Tingyang Wei; Haofeng Wu; Jiao Liu; Zhao Wei; Puay Siew Tan; Yew-Soon Ong· 2026· DOI 10.48550/arXiv.2609.25836

The core problem

Multi-task optimization (MTO) seeks to solve a set of optimization tasks simultaneously by exploiting inter-task synergies. A central challenge is the accurate estimation of inter-task relationships under limited evaluation budgets. When such estimation is inaccurate, knowledge transfer between tasks can degrade performance, a phenomenon known as negative transfer. Conventional MTO methods rely solely on scarce observed data to estimate couplings, which is particularly problematic in few-shot scenarios where each task has been evaluated only a handful of times.

This paper introduces In-Context Guidance Multitask Optimization (ICG-MTO), a framework that leverages numerical foundational models to improve inter-task coupling estimation. Unlike methods that depend only on observed evaluations, ICG-MTO employs a frozen foundational model to infer auxiliary guidance through in-context learning. The framework is designed to provide regularization during the early, data-scarce stages of optimization and to gradually relinquish control as task-specific observations accumulate. The authors instantiate ICG-MTO in two settings: multitask Bayesian optimization (ICG-MTBO) and evolutionary multi

Innovation

The authors evaluate ICG-MTO on synthetic benchmarks and a real-world robot arm control problem. Experiments compare ICG-MTBO against standard multitask Bayesian optimization and ICG-MTO against MFEA-II baselines. The results demonstrate that ICG-MTO effectively mitigates negative transfer and improves optimization performance in few-shot scenarios. Evaluations under different acquisition functions and evolutionary multitasking settings confirm the generality of the framework. The robot arm control problem serves as a real-world validation, showing that the approach transfers to physical systems. The paper reports that the guidance signal provides meaningful regularization early in optimization, leading to better coupling estimates and improved final solutions across tasks.
Multi-task optimization (MTO) seeks to solve a set of optimization tasks simultaneously by exploiting inter-task synergies. A central challenge is the accurate estimation of inter-task relationships under limited evaluation budgets. When such estimation is inaccurate, knowledge transfer between tasks can degrade performance, a phenomenon known as negative transfer. Conventional MTO methods rely solely on scarce observed data to estimate couplings, which is particularly problematic in few-shot scenarios where each task has been evaluated only a handful of times.
This paper introduces In-Context Guidance Multitask Optimization (ICG-MTO), a framework that leverages numerical foundational models to improve inter-task coupling estimation. Unlike methods that depend only on observed evaluations, ICG-MTO employs a frozen foundational model to infer auxiliary guidance through in-context learning. The framework is designed to provide regularization during the early, data-scarce stages of optimization and to gradually relinquish control as task-specific observations accumulate. The authors instantiate ICG-MTO in two settings: multitask Bayesian optimization (ICG-MTBO) and evolutionary multitasking (MFEA-II), demonstrating its generality across algorithm families.

Why it matters

The key insight of ICG-MTO is the use of a frozen numerical foundational model to supply auxiliary guidance when task-specific data are scarce. This addresses a fundamental limitation of conventional MTO: the reliance on limited observations for estimating inter-task relationships. By leveraging in-context learning, the framework can infer predictive relationships among tasks without additional training, making it practical for few-shot settings.

The three-stage design—query construction, guidance inference, and guidance translation—is algorithm-agnostic, allowing instantiation in both Bayesian optimization and evolutionary multitasking. The gradual relinquishing of control as observations accumulate ensures that the guidance does not bias the optimization once sufficient data are available. This adaptive regularization is crucial for maintaining performance across different budget regimes.

Limitations include the dependence on the quality of the frozen foundational model and the design of in-context queries. Future work may explore other algorithm instantiations and broader task families. Overall, ICG-MTO represents a promising direction for integrating foundational models into optimization, bridging the gap between scarce data and reliable inter-task coupling estimation.

Who should read this

CS practitioners and researchers

Opening member content…