Sunday, 14 February 2021

Unity + Oculus Integration on Mac

The Chinese New Year Weekend is too short! I want to spend maximum fun time with the Bean, and get some catching-up-with-sleep-time, but I also want to learn how to make something for Oculus using my Mac alone?? I found several posts online claiming to be able to teach you to set up your Oculus device in 10 minutes. HA I suppose they definitely didn't use a Mac for these speed runs (my Mac has now decided that its new calling is to mimic the hideous sound of an airplane taking off). Still, I persist in valuing the retina display+portablity over practicality and doing everything on my Mac. Will I be forced to retreat back to using a PC again after much frustration? Let's find out!

FUN FACT: According to Rescuetime I spent 33 minutes in total in Unity in order to complete these steps on my 15" Macbook Pro 2019, including all the downloading and importing. The writing of this documentation is probably taking far longer.

The Oculus is a type of Android device so have to check for Android Build Support in the version of Unity I'm using. Just created a 3D project in Unity in the version of Unity I happen to be using.
Unity Asset Store has this default "Oculus Integration". Whilst waiting for that to download, I saw there were so many different integration packages out there for VR and more. Actually got lost browsing all the rather interesting sounding "Tools/Integrations" category on Asset Store. Which ones do the most interesting things? NO CLUE. I guess I will just try Oculus Integration first before actually trying the others.
There are several updates and Unity will need to restart, after which there will be some new Menu items for Oculus like this:
Under Edit > Project Settings > Player > XR Settings > Virtual Reality should be supported.
Another step I would add in is to preemptively remove Vulkan Graphics API, because if you don't, it will throw up an error about XR being incompatible with Vulkan. (Alternatively, I suppose one could go into the OVR scripts which is stopping the build and find the lines where it checks for Vulkan and comment out the checks?)

So I also went to read up on Vulkan Graphics API and what it does - the internet says: "Until now, the mobile graphical interface has been using the OpenGL platform. While the platform was suitable for intense mobile applications like gaming and photography five years ago, the old platform isn’t enough to handle today’s AR/VR intensive applications. It is also not possible to pack in massive hardware in a restrained form factor for running intensive mobile applications. The Vulkan API was developed by Khronos to ensure improved graphical performance with lesser resource usage. The new API has been built from scratch for rendering console quality graphics on existing mobile hardware. What that means is you will be able to enjoy the PC-like graphics on your high-end smartphone". ALRIGHT BUT WE WON"T USE IT.

OH WAIT AS OF 1 FEB 2021 the internets say that Unity now supports Vulkan for Oculus Quest? ¯\_(ツ)_/¯

Ok whatevers. At this point I just removed Vulkan for the time being so I can continue.
Next is to create a developer account and app ID. Now I definitely have mixed feelings about the Facebook integration, which means I have to take several precautions regarding privacy. If I was buying a VR headset only as a casual user, then the issues with forcing users to login with a Facebook account would make me reconsider getting this device. However, the reason I've gotten a Quest 2 is for portability in VR development. Consider all the factors on your own before getting a VR headset device!

Go to http://dashboard.oculus.com/ in a different browser and set up the Developer account.

Connect the Quest to the Mac with the usb cable. Under Build Settings > Android > the Quest should now be available as a device.

Build & Run > and when its done, you can put on the headset and it will start to load your scene. Probably could have used the prefabs to make a scene but there are some demo scenes that came with so I just loaded that first.

Wahooey a demo scene!



NEXT STEPS?

Tiltbrush? Building Tiltbrush which has gone opensource?

How does I workflow???: Workflows: The process flows you should follow

How to screenshot on Oculus Quest 2? Press the Oculus Button + any trigger button. The app needs to have permission to save to storage beforehand.

Where do the screenshots on Oculus Quest 2 go to? Turns out that the Quest is a kind of Android device so on a Mac you have to download Android File Transfer and find screenshots under Oculus > Screenshots.

Saturday, 13 February 2021

VOID: Documentation and Process


VOID started with a bunch of 3D scans. This was the first scan I made whilst waiting at the corner of the void deck for a taxi. The funny thing about this scan was that you got quite a good view of whatever I was looking at, but there's a empty void behind my two legs, where the scanner couldn't reach, which makes it look like sootmark traces in the aftermath of an explosion. There are numerous 3D scanning apps available for iPad Pro and it is easy to export the data as an textured OBJ or glTF/glb and the file can be quickly imported into Unity.

Gameplay images from VOID

These days I spend quite a lot of time waiting for taxis to take me and Beano between places. The timing is such that I frequently end up commuting at daybreak or at night, catching glimpses of dimly lit buildings and sleepy carparks through the darkness. The two destinations I frequent (my house and my parents') are incidentally both what you might consider somewhat complicated places to get to, so there are many opportunities for a grab or taxi driver to become lost or give up the will to find me in the midst of all the one way road systems. So... there is a lot of waiting, during which I've ended up scanning void decks, mining it for material...

Void deck waiting area

When Shih Yun first invited me to show something at OTHERWORLDS, I had several other experiments and drafts that I thought of showing, which I realised over time to be unnecessarily complicated. A good game is not a collage. A game does not necesssarily get better from having more elements in it. I realised that I liked my growing collection of Void Deck scans... and I wanted to make something out of all of them.

HDB Scans from Veerasamy Road

The only weird thing about doing this is that after staring at it in a dream-like state within my game for so long, it is kinda weird to still walk through the same corridors on an everyday basis.
An obvious reference for the work is "In the Night Garden", a show that Beano has been watching a lot of. Some years ago I remember once attending a lecture in the basement of RCA. The details of this lecture eludes me except that the visiting speaker was a new sleep deprived father who had watched a whole lot of "In the Night Garden" recently with their toddler. He apparently had some near-religious epiphany with regards to the philosophical meaning behind "In the Night Garden" and I only have the merest impression that he had meticulously tied it up with an elaborate history of design or critical design or radical italian design pedagogy from the 1970s or whatever it is that RCA design students get lectured to about (and that I was impressed with the mental leaps of the lecture). It thus seems typical that now in my own sleep deprivation of caring for the Bean, I do not recall that useful gem that this forgotten speaker had wished to impress upon us, that rare and useful tidbit that he had gleaned from the watching of "In the Night Garden".

A Screenshot from "In the Night Garden"

Now as I am watching and rewatching pixelated copies of In the Night Garden via youtube on an old and heavily bumpered ipad, I am only left with questions. WHY IS IGGLE PIGGLE ALONE IN A BOAT AT NIGHT? WHERE IS HE SAILING TO? IS THE NIGHT GARDEN THE FEVER DREAM OF AN IGGLE PIGGLE LOST AT SEA? IS IGGLE PIGGLE ALRIGHT? WHERE EXACTLY IS THE NIGHT GARDEN?

Another reference for this work is a scene from another children's show - Steven Universe. Years back I used to catch bits of it and I remember thinking that it was way too childish for me (I probably even said something like "I dont know what people see in it!!!". But when I was on maternity leave I somehow ended up binge watching seasons of it all at one go in the middle of the night. It was when I watched this episode that I was suddenly SOLD on the show, because seeing a nightmare version of the city made me realise that they had done such a good job of painting Beach City to me in the previous episodes. They needed audiences to have built up a good mental map of Beach City before hitting us with a nightmare version of it - otherwise the episode would not have any impact. It is funny that I almost feel kinda nostalgic rewatching old episodes of Steven Universe, because the show now gives me a feeling of Beano being really tiny. I suppose this is the video version of "song that happens to be playing in the background during a defining period of your life". I really love the thoughtful, gentle way in which the show handles topics such as mental health, parenting, and relationships - and I guess I envisioned it being a beautiful cartoon for Beano to enjoy watching with me, one day in the future...

"Rose's Room" (Season 1 Episode 19)

In this episode, Steven discovers that his mother's magical room can fulfill any wish he asks for. After asking it for a bunch of silly things, he is hungry and decides to go get a donut from the shop in Beach City, and he thinks he has stepped out of the room to get a donut, but actually Rose's room has generated a parallel Beach City. Everyone is speaking funny and he runs to the water's edge where he finds the waves are not working properly. Finally after a terrifying time in the city, he realises it is a dream when he tries to eat his donut and it poofs.

Enough of the back story, here is the work!


VOID IN THE CAVE

Shao from DUDE.SG combined all our works into the CAVE setup for the exhibition at Gillman Barracks - a 3 screen kinect based interactive virtual environment! The boat was self-propelling, but it would turn according to the direction of the user's body.
Due to safe distancing measures, the gallery could only accomodate a fixed number of people at one time, so queues outside built up to insane levels. Whilst the show was incredibly short, I heard that there were about 600-700 people who came through each day, so at least we know that many people got to try it!


PRODUCTION NOTES

A few notes here on issues I encountered along the way:

Water in the Boat - Convex Hull

So I wanted to get a boat floating on a sea. I didn't reinvent the sea, no, if you must ask. I did this version with Crest, an existing ocean renderer system for Unity 2019.4.8 and later. A real boat actually displaces the water, so at first when my boat was in the water the ocean rendered into my boat. The fix is to add a convex hull (take the shape of your box and create a shape which is the smallest solid closed convex shape of itself) and then use a script to disable the rendering of the water where the convex hull is located inside the boat. No more water inside boat.

Resolution

One thing I’ve blithely not comprehended before was that on a MacBook with Retina display, the pixel density is twice that of the norm, therefore instead of 72dpi it is actually 144 dpi, meaning for every 1px there is actually now 4px (2px x 2px), hence when I take screenshots/screencaptures that do not account for the embedded dpi setting of 144dpi, they will come out twice as big. A crucial issue when trying to screencapture my game as a gameplay video…

Colour / Dropped Frames / Understanding Unity Profiler

Monosnap is the trusty screenshot app I've used for several years now. Using it meant that I never had to really think about how I really wanted to compress things, because I just trusted it to quickly crop and compress things for me (whether image or video). However, most of what I've been screenshotting with it so far have been things like annotations of images and lectures, for which colour is not much of a big deal. Well, my lazy days are ending. Using it to capture a gameplay video this time around showed me that maybe I need a better long-term solution for gameplay screencapture (possibly even an internal game recorder?) because (1) there is something going on with the colour (probably to do with conversion from 10 bit colour to 8 bit colour, which flatten my subtle purples to black), and (2) there is also something going on with dropped frames, which I am guessing is possibly my macbook pro's gpu being not up to the task of, so it all points to (3) that I probably need to watch more tutorials on how to properly use the Unity profiler to improve the performance of my game... ¯\_(ツ)_/¯




COMING SOON: VOID IN VR?