BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability
Researchers have introduced BONSAI, a new default-aware Bayesian optimization (BO) policy designed to enhance the simplicity and interpretability of black-box function optimization. Standard BO techniques often push weakly relevant parameters to search space boundaries, complicating the distinction between significant and spurious changes. BONSAI addresses this by pruning low-impact deviations from carefully engineered default configurations while explicitly controlling acquisition value loss. The method is compatible with various acquisition functions, such as expected improvement and GP-UCB. Theoretical analysis demonstrates that BONSAI maintains the same no-regret property as vanilla GP-UCB and provably recovers the relevant-coordinate set at zero acquisition cost under specific assumptions. Empirical results across real-world applications indicate that BONSAI significantly reduces the number of non-default parameters in recommended configurations without compromising optimization performance. Furthermore, it offers superior computational efficiency, averaging only 1.5 times the candidate-generation cost of standard BO, compared to 7-34 times for prior sparse-BO methods like IR, ER, and SEBO.
Wire timeline
BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability
Researchers have introduced BONSAI, a new default-aware Bayesian optimization (BO) policy designed to enhance the simplicity and interpretability of black-box function optimization. Standard BO techniques often push weakly relevant parameters to search space boundaries, complicating the distinction between significant and spurious changes. BONSAI addresses this by pruning low-impact deviations from carefully engineered default configurations while explicitly controlling acquisition value loss. The method is compatible with various acquisition functions, such as expected improvement and GP-UCB. Theoretical analysis demonstrates that BONSAI maintains the same no-regret property as vanilla GP-UCB and provably recovers the relevant-coordinate set at zero acquisition cost under specific assumptions. Empirical results across real-world applications indicate that BONSAI significantly reduces the number of non-default parameters in recommended configurations without compromising optimization performance. Furthermore, it offers superior computational efficiency, averaging only 1.5 times the candidate-generation cost of standard BO, compared to 7-34 times for prior sparse-BO methods like IR, ER, and SEBO.
cs.AI updates on arXiv.org