Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks

Researchers Cheol Young Park and Shou Matsumoto have introduced the Free Energy Manifold (FEM), a novel score-trained conditional energy model designed for inference in hybrid Bayesian networks containing both discrete and continuous variables. The model represents conditional factors as energy landscapes over learned embeddings, facilitating posterior evaluation and generative sampling. A key contribution is the identification of the 'mode-bridge artifact,' where standard models create low-energy ridges between separated modes, leading to overconfident posteriors. To address this, the authors propose 'valley regularization,' an off-data calibration term that ensures near-uniform posteriors in these regions while maintaining accuracy on observed data. Benchmark tests on synthetic multimodal hybrid networks and UCI datasets demonstrate that FEM significantly reduces KL divergence compared to classical baselines and vanilla conditional Energy-Based Models. The study highlights FEM's effectiveness in multimodal and compositional inference tasks, though it notes that discriminative classifiers remain superior for closed-world classification scenarios.