What Being Ripped Off Taught Me

Summary

In Spring of 2024, a good friend contacted me saying he’d heard of an opportunity to help finish an augmented reality bus tour of a Beijing park. He’d gotten the impression the California-based client were in a pinch, and said they’d mentioned by way of referral the names of some folks we both knew and respected. In the past, I’d consulted on AR bus bids for both The Mill and IDEO, and from those I was well aware how many hard problems AR buses involved. I was very curious to find out how these folks had addressed them, and on our introductory call I told them as much. They demurred. This was my first warning not all was right. We spoke briefly about the nature of the project and their need for help, and agreed I’d immediately try to get a work visa and come out to China for as long as I could spare - we determined it’d be a month. Fortunately I found I still had a valid visa from a previous project, so I was good to head out immediately. When I arrived I found the project in shambles. Multiple very junior developers were touching (binary, TouchDesigner) code and deploying straight to production via thumb drive, with zero version control. In fact, they didn’t know what version control was. They were attempting to pull off AR effects on the transparent OLED windows of the bus without accounting for lens distortion, field of view, parallax, occlusion, etc., and were frustrated and mystified when things didn’t appear to line up. They were completely naive to what depth and scale cues are and how to deploy them. Their gyroscopes had an axis flipped, and the pitch of the bus was producing an opposite reaction in the virtual content. GPS was highly unreliable (because China) and there was no workaround in place. Their render pipeline was drawing everything - including their renders - to fullscreen quads, which were then used for basic compositing with alpha and re-rendered - all 35+ and counting layers, using alpha-to-transparency. Their (consumer, air-cooled) gaming PC’s...