Easy-but-effective domain sub-similarity learning for transfer regression
Abstract
Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions,${T}_{*}$, that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain validGPs using${T}_{*}$($GP_{T_{*}}$) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of$GP_{T_{*}}$, one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of$GP_{T_{*}}$on the sub-similarity capture and the transfer performance.
Authors: Pengfei Wei, Ramón Sagarna, Yiping Ke, Yew-Soon Ong
Published in: IEEE Transactions on Knowledge and Data Engineering (2020)