Testing a Large-Scale VR Multiplayer
How VRROOM tested a live VR concert with 55 simultaneous attendees. A series of test rounds with varying headset requirements, target audiences, and build sharing logistics.
About VRROOM
VRROOM is a production studio that produces high-quality immersive shows, cultural events, and festivals, hosts them on their XR social platform, and distributes them live and in replay.
The Challenge of Testing a VR Multiplayer
VRROOM wanted to thoroughly test their VR multiplayer platform before the highly anticipated closed alpha launch featuring popular French artists such as Web7, Le Juiice, Maxence, and DJ m1n0t0r. The tests involved hosting live concerts. The objective was to conduct multiple test sessions, gradually increasing the number of participants from 15 to 55 to simulate real-world usage scenarios.
The primary challenge laid in coordinating a multitude of factors, including meeting profile and headset requirements, ensuring task completion, scheduling and attendance management, build logistics—flexibility to adjust the start time of the test in real-time, provide instructions, and request testers with specific headsets and profile based on the build launched (e.g., .APK, .EXE, standalone or PC VR, etc.)—which meant testers have to be promptly notified, replaced, or invited in real-time.
Ensuring a specific ratio of headsets, including and accommodating both, experienced and casual testers, and managing recurring and new participants further increased the complexity. VRROOM needed a solution for testing a VR multiplayer to complete all tests following the requirements.
Solution and Outcome
VR Oxygen successfully addressed the following:
- Headsets and Ratio.
Accommodating varying headset requirements for different test rounds closely keeping to a required ratio—a varying per test round percent for standalone and PC VR headsets. Standalone included a ratio of Quest 2, Quest 1, Quest Pro, Pico 4, and PC VR included a ratio of HTC Vive, Valve Index, Pico 4, Quest +Link Cable/AirLink/Virtual Desktop. - Targeted Audience.
Finding specific profiles and incorporating them as per requirements: Starting the tests with experienced VR users only and adding casual users in a predetermined ratio, also the ratio for geolocation, recurring, and new testers. - Efficient Multiplayer Test Scheduling.
Facilitating the scheduling and organization of test rounds, enabling VRROOM to gather all participants on specific days and times for three-hour-long sessions. - Build Logistics Notifications.
Real-time notifications, instructions, and requests for testers with specific headsets and profiles. Testers were promptly notified, replaced, or invited as per changing in-real-time requirements. - Task Completion and Deliverables.
Ensuring the completion of specific tasks and deliverables. Deliverables included a recording of the first user experience (ensuring the video and audio length requirements are met) and a post-play survey for each tester, providing consistent test results for all test rounds. VRROOM team members attended and moderated all live tests in VR. The post-play survey was created using the question library. The suggestions based on the goals questions were edited by the VRROOM team.
- Reliable Attendance Management.
Missing participants were promptly replaced with suitable profiles in real-time, guaranteeing the desired range for the number of attendees. The fluctuations by 1-5 testers (varied for each test round) were within acceptable limits, considering the complexity of the task.
“These tests have been very useful for us to close the development of this stage of the app.”
- Antony Vitillo, CTO
VRROOM successfully conducted a series of large-scale VR multiplayer tests, getting actionable results for this development stage.
The outcomes provided VRROOM with the necessary feedback and insights to refine their platform before the closed alpha launch. Testing a VR multiplayer is a complex task, and if done right, it will provide invaluable information about performance and player experience in real-life situations.
Image credits: VRROOM