Research
My research sits at the intersection of virtual reality, human-computer interaction, multimodal sensing, and applied training systems. I design and evaluate AI-enabled immersive experiences that make virtual agents, complex procedures, spatial behavior, and social interaction easier to understand and practice.
I build real-time systems for multimodal interaction with embodied agent groups in VR. A Unity/C# client connects to a Python/FastAPI backend through WebSockets, integrating speech recognition, LLM orchestration, speech synthesis, and generated humanoid motion. The work explores verbal and spatial interaction alongside group-level coordination.
I developed a 3D compass interface in Unreal Engine to combine symbolic input and contextual cues for VR password entry. A within-subject, multi-session study with 32 participants evaluated entry efficiency, recall, subjective memorability, and resistance to observational attacks. Published at IEEE ISMAR 2024 with co-first authorship.
Y. Huang, D. Zhang and E. S. Rosenberg, "Direction-Based Authentication: Combining Symbolic Input and Contextual Cues for Virtual Reality Password Entry," 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bellevue, WA, USA, 2024, pp. 681-689.
Co-first author.
Y. Huang, D. Zhang and E. S. Rosenberg, "DBA: Direction-Based Authentication in Virtual Reality," 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Shanghai, China, 2023, pp. 953-954.
Equal contribution.
COVID-Vision is a virtual reality experience designed to encourage mindfulness of social distancing in public spaces. The project uses spatial feedback and embodied perspective-taking to make invisible interpersonal distances easier to perceive.
D. Zhang et al., "COVID-Vision: A Virtual Reality Experience to Encourage Mindfulness of Social Distancing in Public Spaces," 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, 2021, pp. 697-698.
Working with nursing education specialists, I developed two interactive Unity scenarios with patient avatars and real-time verbal feedback. Iterative expert and student evaluations informed the design. The paper is available as a preprint and is under review at ACM Transactions on Computing for Healthcare.
D. Zhang, C. Bradley, C. Kissell, C. Hutton Pospick, S. Yong, Q. Jin, T. Nie and E. S. Rosenberg, "Iterative Design and Evaluation of a VR Nursing Training System: Insights from Clinicians and Student Learners," arXiv:2610.19152, 2026.
Under review at ACM Transactions on Computing for Healthcare.
Working with two orthopedic experts, I built a Unity prototype for a specialized knee surgery procedure and modeled more than 12 medical instruments in Blender. Interactive instruments and anatomy, learning-summary panels, and automatic video pauses at eight critical learning points support guided practice. Six surgeons evaluated the prototype.
This NSF-sponsored work compares video conferencing and VR over time to understand how communication technology shapes group processes, team performance, individual well-being, and social need fulfillment.