InterDance: Reactive 3D Dance Generation with Realistic Duet Interactions

Abstract

Humans can perform a variety of interactive motions, among which two-person dance is one of the most challenging interactions. However, in terms of computer motion generation, current work is still unable to generate high-quality interactive motion, especially in the field of duet dance. On the one hand, this is caused by the lack of large-scale high-quality datasets. On the other hand, it arises from the incomplete representation of interactive motion and the lack of fine-grained optimization of interactions. To address these challenges, we propose a duet dance dataset that significantly enhances motion quality, data scale, and the variety of dance genres. Based on this dataset, we propose a new motion representation that can accurately and comprehensively describe interactive motion. We further introduce a diffusion-based algorithm with an interaction refine guidance strategy to optimize the realism of interactions progressively. Experiments demonstrate the effectiveness of our dataset and algorithm.

InterDance Demo

Data Examples (Quick Scan)

This section showcases the dances in the InterDance dataset.
Rumba
Cha Cha
Waltz
Jazz
Shenyun
Jive
Tai
Uighur
Hantang
Hiphop
Dunhuang
Kpop
Urban
Miao
Samba

Reactive Dance Generation

This section showcases the dances generated by our method.
The reative dance generation task need to generate the follower's dance from given music and leader's dance.
Rumba (React)
Waltz (React)
Sumba (React)
Shenyun (React)
Samba (React)
Waltz (React)
Rumba (React)
Urban (React)
ShenYun (React)
Waltz (React)
Sumba (React)
Waltz (React)

Duet Dance Generation

This section showcases the dances generated by our method.
The duet dance generation need to generate two-person's dance from given music.
Waltz (duet)
Rumba (duet)
Chacha (duet)
Waltz (duet)
Waltz (duet)
Rumba (duet)
Waltz (duet)
Waltz (duet)

Long Reactive Dance Generation

This section showcases the long dances generated by our method.
Urban (long)
Waltz (long)

Repaint results w/o leader label

we applied the diffusion repainting technique and directly used the reactive dance generation model to generate the follower dance under the control of leader motion (directly testing and without the leader contact label).
Waltz (repaint)
Waltz (repaint)
Rumba (repaint)
Rumba (repaint)

Reactive results trained w/o leader contact label

we retrained our network without providing the leader contact label to generate the follower dance.
Waltz (w/o leader contact)
Waltz (w/o leader contact)
Rumba (w/o leader contact)
Rumba (w/o leader contact)

License

The dataset and code will be officially released alongside the camera-ready version of the paper, for academic research purposes only.