THEATER OF FUTURE PUPPETRY
Type
Interactive System
Role
XR Software Engineer
Date
May 2022 - Jan 2025
Client
National Science and Technology Council (NSTC)
Credits
Production: Chun-Cheng Hsu Lab
An immersive puppetry system that integrates VR gesture recognition, multi-user mobile interaction, and traditional Taiwanese glove puppetry. It enables audiences to engage with puppetry in innovative ways while contributing to cultural heritage preservation.
2025 — DiversePuppetry, Posters, SIGGRAPH, Vancouver Convention Centre, Canada
2025 — Theater of Future Puppetry, Student Innovation, SXSW, USA
2024 — Theater of Future Puppetry, XR Program, SIGGRAPH Asia, Tokyo International Forum, Japan
2024 — Future Puppet Theater, FUTEX, Taipei World Trade Center, Taipei
2024 — Heart of the Machine, Ars Electronica, Austria
In VR, users control five classic puppet roles via Quest hand tracking. The gesture set, inspired by Budaixi and co-designed with masters, is implemented with the Meta SDK and mapped to real-time puppet actions.
To bring puppetry into public spaces, we created a mobile-based system with immersive projection that enables multi-user participation. Visitors scan QR codes to select roles, animate puppets, and interact collaboratively in real time.
Each platform reveals unique ways to experience puppetry. Building on these strengths, we developed DiversePuppetry, an asymmetric system that integrates VR-HMD, MR-HMD, and VR-CAVE to animate digital puppets across devices.
This approach enables collaborative performances and reimagines the future of puppet theater.
