As a long-time web developer and non-gaming VR fan I'm really excited about the potential for the web on virtual reality platforms like the Oculus Quest. Last week I spent a few hours toying with my 2D tetris clone on the Oculus Quest browser and actually got it installed it as a Progressive Web App on my headset. I'm excited to show off how I got this to work and what I learned in the process.
Oculus Quest is Essentially an Android Device
The Oculus Quest is basically an Android smartphone that you strap to your face. To get started developing for the Quest, even on the web, you're going to want to download Android Studio and the Android SDK Platform Tools. You'll be using the adb command from time to time to confirm your device is properly connected, install your PWAs and make other advanced configuration changes. You also will need to enable developer mode for your device, which you need to do from the Oculus app on your phone. There's also another step in the Oculus settings menu where you'll need to enable USB debugging.
Oculus provides some desktop software called Oculus Developer Hub that simplifies a number of key developer tasks like verifying your connection, disabling the guardian, casting and taking screenshots.
It’s definitely a lot of steps, but after you've got all of this setup you're well on your way to building Oculus VR apps in 2D using web technology. It's exciting!
Browsing the 2D Web in VR
When you open up the Oculus Browser for the first time you might notice a few things that are unique about it compared to a normal web browser. The first is that it's capable of 3 browser windows side by side. So you're confronted with this 180° wall of browser windows which measures over 3000pixels in width. The default width of each window according to the oculus browser specs is unique at 1000px wide by 505px high. And as of right now you can resize them, but only the width. You’re stuck with that strange height. (Edit: Width/Height Resizing is now available as of Dec. 13, 2021.)
Try to pull up a few websites. If it's something interactive like my tetris game there's a good chance it won't quite work!!
If on the other hand you pull up a standard news site, blog or even the something optimized like the React Three Fiber Demo App they should all look and work great.
Debugging Local Code
Assuming you're building your app on your local computer you probably want to see those updates immediately in the oculus browser. To do that you need to setup a reverse proxy which forwards all traffic from a port on your Oculus device to one on your computer:
adb reverse tcp:3000 tcp:3000
With that you should be able to enter in URLs like http://localhost:3000 in the oculus browser and have them work perfectly, reading files off of your computer.
If you're using something like codesandbox or glitch to host your files it can be really helpful to use this feature of Oculus Developer Hub which lets you enter in a URL and have it open automatically on your headset.
Now comes the mind blowing part. You can inspect your oculus browser window from inside of Google Chrome. You simply go to
chrome://inspect/#devices and click "inspect" on your device instance. It really feels like magic.
I mostly used console.log to log my event listeners, but as far as I can tell the full JavaScript debugger along with the network and element inspectors. Because going back and forth between your computer and your headset is obviously kind of a pain, I found testing by interacting with the little preview app inside of inspector window actually saved a lot of time.
UX & Web API Considerations
✅ Now you have your app showing up in the Oculus browser and you're able to update and debug that app on your local computer. Here are some basic tips for optimizing that app for running in VR:
- Use the PointerEvents API for user input
- Make buttons and text Larger
- Optimize for a wide screen
My initial tetris implementation relied on keyboard input to move things around. When first porting it over I started using mouse events like mousedown and mousemove and was surprised when they didn't work. I shouldn't have been surprised though. The Oculus Browser is running on an Android device. Instead of mouse events it supports Touch Events and the slightly cleaner PointerEvents API. Pointer Events give you have a single event for both touch and mouse events. They're fully supported in React and seem a little easier to work with than touch events, which support input from multiple touches simultaneously.
Once you get your user input working properly make sure to make your interactive elements larger. As mobile web browsing began to take off a decade ago, both Apple and Google provided UI guidelines for how users interact with their devices. One of the main ones was to give a lot of padding to buttons & other controls you expected the user would touch. This was to make sure when someone's finger didn't quite make the target your app would give them what they wanted anyway. This is even more important in VR where your controllers are in a constant state of motion. Make those inputs larger and given the blurry text we still have in VR, make your text larger as well.
My last realization is to consider the screen real estate you have in the Oculus Browser. You're dealing with 1000px wide, but only 505px high. It's a fairly wide screen, but it's really short. Scrolling in the Oculus Browser works fine, but I'd personally recommend designing for wide screen without having to scroll. Certainly for most games or more interactive apps you don't want to have to scroll to see some important part of the action. Think about how you can stretch your app out. Many tablet apps are designed to take advantage of space in landscape mode and I'm confident we'll figure out how to do this well for VR.
More To Learn
My tetris clone still isn't fully optimized for VR, but I made the buttons a lot bigger and migrated to use the Touch Events API so that pieces can be moved around as they are falling. I also got it installed as a PWA so I can run it next to other apps in my library. My next steps are to migrate to the Pointer Events API and to figure out how to optimize for all of that horizontal screen real estate!
Once I get that sorted out there are definitely a few more questions I want to dive into. Do I need the WebXR Gamepads API to capture the various buttons and trigger inputs from the oculus touch controllers? (Follow up: nope, they can't be accessed outside of immersive-vr mode.) Could I make a super wide game that spans 3 browser windows using postMessage or the Web Storage API? And then beyond that, is a 2D web for VR even a thing worth pursuing? How hard would it be to use something like a-frame or React Three Fiber with use cannon to make fully immersive 3D world? People are doing it, so why not you and me?
There are so many technologies generating enthusiasm right now in the world of web development, but I wouldn't sleep on building web apps for VR! Meta has sold over 10 million Quest 2 devices already and that growth seems to be sustainable. This is happening. What role is the 2D web going to have in virtual reality and the so-called metaverse? What about WebXR and the 3D web? It's up to us to build the future we want to see. So start building!
- https://techcrunch.com/2021/10/28/facebook-opens-oculus-store-to-2d-progressive-web-apps/
- https://www.theverge.com/2021/11/16/22785469/meta-oculus-quest-2-10-million-units-sold-qualcomm-xr2
- https://developer.oculus.com/documentation/web/browser-remote-debugging/
- https://reactjs.org/blog/2018/05/23/react-v-16-4.html
- https://developer.apple.com/design/human-interface-guidelines/ios/user-interaction/gestures/
- https://www.w3.org/TR/webxr-gamepads-module-1/
Top comments (0)