Los Angeles-based technologists and content creators All of it Now (AOIN) are revolutionizing the way brands, agencies, entertainers and events engage their audiences. With a range of services spanning from design and previsualization (previs) to virtual location scouting and augmented reality/extended reality (AR/XR) consultation, AOIN is at the forefront of immersive experiences. Their latest project involved creating an AR-enabled presentation of Disney’s Encanto at the iconic Hollywood Bowl. To bring this grand vision to life, AOIN turned to Sony’s Spatial Reality Display. By utilizing Sony’s Spatial Reality Display, AOIN crafted highly realistic, three-dimensional content that captivated audiences, all without the need for special glasses or VR headsets.
AOIN became aware of Sony’s Spatial Reality Display after seeing it showcased in the company’s booth at the National Association of Broadcasters (NAB) Show. Berto Mora, AOIN’s chief technology officer, recalls many industry colleagues telling him to check out Sony’s new display, and he wasn’t disappointed. “It was one of the coolest things I’ve seen,” he says.
Sony’s Spatial Reality Display Helps Streamline Workflows
The main selling points for Mora were the Spatial Reality Display’s integration with Unreal Engine, its capacity to showcase and visualize 3D content without wearing glasses, the ability to use it for virtual production, and the opportunity to showcase massive content in a realistic way from a tabletop device. Shortly after receiving a demo, AOIN bought a unit to use in-house.
Danny Firpo, cofounder of AOIN, used the Spatial Reality Display to help streamline their pre-production workflow. Speaking of the Encanto show at the Hollywood Bowl, he explains, “First, we used the Spatial Reality Display to help visualize the Hollywood Bowl venue. Additionally, because the content is motivated by projection, many of the effects used in the AR broadcast were particle-based. Sony’s display provided visuals in both a 2D and a 3D space, enabling a better understanding of the paths that the particles were taking, which allowed us to spatialize and contextualize them.”
Beyond a better understanding of the particles, the Spatial Reality Display was also beneficial in finding camera positions. Firpo continues, “It was really helpful to see the Bowl in a 3D space and use that for identifying camera positions and cable paths, collaboratively. That 3D environment served as a vital pre-production tool.” He adds, “Using this display provided us with some special moments. We coordinated closely with the director of photography and individual camera operators to show them the path in Unreal within the Hollywood Bowl, which was helpful for planning the cameras’ paths with the jib. We were able to see dramatic plans as a result of seeing the camera movements in the 3D space.”
AR Visualization Tool
Nicole Plaza, AOIN’s executive producer, expands on the abilities that the Spatial Reality Display offers. She said, “Augmented reality is new for a lot of people, so having a tool that allows directors and others to visualize it and gives them an understanding of what we’re doing is important. It allows them to be immersed in that 3D space and understand our work and workflows more quickly.”
Mora adds, “We work in a lot of 3D spaces and 3D worlds through the engines, specifically Unreal Engine. It’s difficult to convey what you’re looking at through a flat, two-dimensional screen. The beautiful thing about the Sony’s Spatial Reality Display is that you can now visualize that content in 3D and better understand it, instead of asking someone to imagine it in 3D. With that comes the ability to play with depth and the perception of 3D with the human eye. And that all makes the content more valuable.”
Firpo also appreciates the presentation abilities that the Spatial Reality Display offers to customers. “The display allows us to literally be able to look around and navigate the spaces intuitively. It creates a transparent technology layer for those who aren’t used to Unreal environments. In addition, the Spatial Reality Display is very easy for us and for clients to use. You don’t have to use any buttons to navigate it. And at the end of the day, everyone appreciates the technology and loves seeing the content.”
AOIN Uses Sony’s Spatial Reality Display, Earning Rave Reviews
Speaking of clients, AOIN has received positive responses to the Spatial Reality Display from creatives. “Production designers often deal with the 2D space, so for them to see their designs in a 3D space before a final version is completed is helpful to provide context and spatial reference. To also see those AR effects interacting with production design pieces is a very special thing for a lot of these production teams and we’ve received some really great feedback. Beyond just seeing the final output or mockups, we can provide insight into the whole process,” explains Firpo.
He adds, “The great part is, a lot of the artists that work on the content also get to go on-site and see it live. I think it’s really fulfilling, and it creates a positive reinforcement that artists can see their work first in a 2D space, then the 3D space with the Spatial Reality Display during previs and pre-production and then finally live on camera during the broadcast.”
Sony’s Spatial Reality Display and its Features
The Spatial Reality Display is comprised of several features including a real-time rendering algorithm, a micro-optical lens, easy integration and a proprietary high-speed vision sensor. Mora shares that one of his favorite aspects of the display is the eye-sensing technology. “The gaze tracking of your eyes lets you know this isn’t a conventional display. The real-time perspective shift is a huge selling point that makes it much more interesting. This feature complements XR environments nicely.”
Related: How Integrators Can Benefit from Virtual Reality
Similarly, AOIN appreciates the ability to view their content unencumbered by glasses. “The display looks more realistic and presents content more accurately when you’re creating it. It provides a different effect when compared to wearing glasses,” Mora explains. Firpo adds, “As somebody who already wears glasses every day, there’s a fatigue layer and added friction. The weight of the glasses on your face limits the amount of time you can use them. It can also cause strain on your neck, and glasses can hurt your face and make you sweat. There’s also the potential for dirt and smudges to present themselves, which can degrade the viewing experience and affect the image quality.”
Firpo explains that his and his clients’ familiarity and experience with the Sony brand also contributed to AOIN’s decision to choose the Spatial Reality Display. He says, “The Sony brand is trusted by us, our clients, producers and camera operators. Using a Sony display adds a level of confidence, as well as compatibility since we’re often using Sony’s cameras on-site. To have a Sony pipeline that uses multiple different products like the VENICE and Spatial Reality Display gives us peace of mind that the color reproduction is correct and accurate.”
Endless Possibilities
As AOIN continues their use of the Spatial Reality Display they’re excited by the opportunities it presents not only for preproduction, previs and AR but the additional potential it will provide for in-camera virtual effects for longer-form television and film, as well as virtual production – areas where the company, and the industry, are rapidly expanding. With the promise that digital characters and metahuman interactions hold, the possibilities are endless. Sony’s Spatial Reality Display will continue to help AOIN socialize these concepts to their clients in a highly realistic and impressive fashion as they continue to experiment with technology and push the boundaries of visualization.
