ChladniSonify: Real-Time Visual-Acoustic Mapping for New Media Art
Researchers have introduced ChladniSonify, a novel real-time visual-acoustic mapping method designed to enhance new media art creation by addressing the subjectivity and technical limitations of existing audio-visual tools. Traditional methods often suffer from high simulation barriers, offline computing constraints, and uncontrollable mapping rules. To overcome these challenges, the team developed a system based on Kirchhoff-Love plate theory, utilizing a paired dataset calibrated via ANSYS finite element simulation. The core technology employs a lightweight Convolutional Neural Network (CNN) with CBAM to classify slender nodal lines in Chladni patterns with high precision. Implemented as an end-to-end system using Python and Max/MSP, the tool maps recognized patterns to corresponding sine wave frequencies. Performance tests demonstrate exceptional usability, achieving 99.33% classification accuracy with only 7.03 ms inference latency. The mapped frequencies match theoretical values with zero deviation, and the average end-to-end latency remains under 50 ms, successfully meeting the rigorous demands of real-time interactive art installations. This work provides a reproducible engineering prototype for artists and developers.
Wire timeline
ChladniSonify: Real-Time Visual-Acoustic Mapping for New Media Art
Researchers have introduced ChladniSonify, a novel real-time visual-acoustic mapping method designed to enhance new media art creation by addressing the subjectivity and technical limitations of existing audio-visual tools. Traditional methods often suffer from high simulation barriers, offline computing constraints, and uncontrollable mapping rules. To overcome these challenges, the team developed a system based on Kirchhoff-Love plate theory, utilizing a paired dataset calibrated via ANSYS finite element simulation. The core technology employs a lightweight Convolutional Neural Network (CNN) with CBAM to classify slender nodal lines in Chladni patterns with high precision. Implemented as an end-to-end system using Python and Max/MSP, the tool maps recognized patterns to corresponding sine wave frequencies. Performance tests demonstrate exceptional usability, achieving 99.33% classification accuracy with only 7.03 ms inference latency. The mapped frequencies match theoretical values with zero deviation, and the average end-to-end latency remains under 50 ms, successfully meeting the rigorous demands of real-time interactive art installations. This work provides a reproducible engineering prototype for artists and developers.
cs.AI updates on arXiv.org