Curvature-Aware Captioning: Leveraging Geodesic Attention for 3D Scene Understanding

Researchers have introduced a novel framework titled Curvature-Aware Captioning to address significant limitations in current 3D scene description methods, particularly when processing sparse point cloud data. Existing approaches using Euclidean embedding spaces often fail to simultaneously preserve fine-grained local geometric details and model complex global semantic hierarchies, resulting in inaccurate localization or shallow descriptions. This new method integrates non-Euclidean geodesic attention mechanisms to resolve the conflict between localization and contextualization. Specifically, it employs self-attention within Oblique space to enforce dimensional homogeneity and establish long-range dependencies, while bidirectional geodesic cross-attention in Lorentz space models hierarchical semantic relationships. Theoretical analysis confirms that the curvature complementarity between the Oblique manifold and Lorentz hyperboloid ensures feature stability and preserves inherent hierarchical structures. Extensive experiments on the ScanRefer and Nr3D benchmarks demonstrate that this approach achieves state-of-the-art performance, offering significant improvements in both object localization accuracy and the richness of scene descriptions, with potential applications in robotic navigation and augmented reality.