Sony Spatial Reality Display Reviewed

In the late fall of 2020 Sony announced the release of their Spatial Reality Display using their award-winning Eye-Sensing Light Field Display (ELFD) technology.

I was able to get my hands on a demo display and Lenovo workstation to get a sense of real-world use using a number of sample applications authored in Unity.

If you have not heard about or seen this display, it is about as close as you can get to a production "holographic" display as technology will allow.  The unit itself is 15.6” and resembles an iPad mounted at a permanent 45-degree angle.  Near the top of the display is a camera that performs eye-tracking using its high-speed vision sensor.  Overlaid on top of the 4K display is a Micro-Optical lens that divides the left and right eye to allow for stereoscopic viewing.

The net result is a display that simulates 3D depth and parallax on what is otherwise a flat 2D display.  Moving your head side to side or in and out, allows you to look around the 3D environment and observe that parallax effect. You can also interact with the model just as you would in a 3D application, but this is more a function of the design of your application.  Experiences for this display are developed using either Unity or Unreal Engine. 

For those looking for a detailed breakdown of the display specifications, I have included the most important ones below.

Weight: 10.2lbs
Viewing Distance: 30-75cm
Viewing Angle: Up 20 degrees, down 40 degrees, left and right 25 degrees
Number of users at a time: 1
Display size: 15.6”
Luminance: 500 nits
Contrast: 1400:1
Gamut: Adobe RGB approx. 100%
Colour Temperature: 6500K
Panel Resolution: 3,840 x 2,160 pixels
Audio: 5.5 W speaker
Connectivity: 1 HDMI input, 1 USB control terminal, 1 Power DC IN.
Supported Game Engines: Unity 2019.4 (recommended), 2018.4, 2020.1 / Unreal Engine 4 - 4.25 (recommended), 4.24

Below you can see the minimum and recommended hardware required to use the SR Display. Sony’s own documentation also indicates that even under the recommended operating environment, the frame rate may drop significantly depending on the content.  Your authored environment is required to achieve a minimum frame rate of 60 FPS in order to ensure a smooth viewing experience. 

The display has three key features that allow it to do what it does.  A high-speed vision sensor, a real-time video algorithm and a micro-optical lens.

1. High-speed, high-precision real-time sensing technology
High-speed vision sensors and eye-recognition technology always detects the position of the viewer's eyes. It detects the position of each of the left and right eyes in real-time, not only horizontally and vertically, but also in-depth.

2. Real-time video generation algorithm
Based on the positional information of the user's eyes, the light source video that comes out of the display panel is generated in real-time from 3DCG data.

3. Micro-optical lenses
Our unique micro-optical lens, which delivers stereoscopic images to the left and right eyes, is attached to the entire panel surface with ultra-high precision, enabling natural naked-eye stereopsis.

Using the SR Display requires a fair bit of horsepower as the GPU and CPU must render 4K 60p footage.  The Lenovo workstation that we were provided was a Legion T5 28IMB05 with a Core i7-10700 CPU @ 2.9GHz, 16GB of RAM and a GeForce RTX 2070 Super GPU.  In testing the examples provided, almost all of them maxed out the CPU and GPU to 100% trying to keep up and you could hear all of the fans spinning up.  It’s difficult to get a sense of how far you could push the polygon limits and overall scenes, but all of the demos in this review were very processor-intensive.

As mentioned earlier currently Unity and Unreal are supported as authoring platforms for the SR Display. 

Like VR head-mounted displays (HMD), care does need to be taken when authoring experiences for users to avoid motion sickness or discomfort. One of those techniques is to ensure all of your assets and lighting fall within the SR Display "View Space".  If you imagine a virtual box projecting from the angled screen, this is your authoring environment.  The display also ships with parts that attach to the screen to further enhance this space and improve the sense of immersion.  However, because of this confined authoring space, most experiences feel like you are viewing a miniature model, rather than an immersive environment that you might expect while using a VR HMD. I did not find this to be a negative, however, it does need to be noted given the size of the display. I imagine as the technology matures and screen size can increase, that miniature feel will be reduced.  Authors are also cautioned against moving the scene camera around significantly as this can introduce nausea and eye fatigue. 

There is not a lot of information about what size of environment can be authored on the display beyond the advice to ensure you maintain 60 FPS at 4K.  All of the experiences I recorded in the video at the beginning of this review appeared to render without issue, less the cross-talk I will speak about next.

In all of the experiences I tried, all displayed some level of what is called Cross-Talk.  Because the 3D image is being divided for the left and right eye using the micro-optical lens, you do get a level of artifacting or ghosting that occurs as you move your head around the scene.  The level of cross-talk seems to depend on how quickly you move your head and the type of scene.  The developer documents cite large differences in colour or brightness both in front and behind the scene.  Likewise, the same thing will occur with large differences in luminance or saturation between materials.  In all of the videos I recorded, those cross-talk artifacts are greatly exaggerated due to do camera recording, however, it does also give a sense of what that effect looks like when it does happen.

Both Unity and Unreal have special scripts to try to minimize this impact. The level of compensation however is a balancing act between the reduction of cross-talk and the processor demands.

If you want to delve more into the technical requirements and authoring limitations, Sony has created a fairly in-depth developer website with a lot of the information required.

Overall the display is solidly built and feels like a high-end piece of equipment.  The only critique I would offer is the add-on pieces to accentuate the view space.  The two side triangle pieces are held on by magnets and feel a bit flimsy and the bottom horizontal piece in front of the display is just plastic and clips to the front of the display quite loosely. The one design flaw is the metal bar that secures across the top of the display.  It feels very poorly executed. I assume it’s to hold the triangle pieces in place, but its position also blocks about 10 pixels of the top of the screen. This might sound trivial, but that’s also where all of the menu bars happen to reside when you are in Windows. In order to close a full-screen window or navigate a menu bar, you need to tilt your head below the bar or lift the bar up.   I would also not want to move the display around without removing all of these extra pieces, as they would most definitely fall off.  

The display is quite clear and has nice colour, but the micro-optical lens does reduce the quality of the 4K LCD panel that is below.  For anyone that has tried an earlier generation VR HMD, I’d equate the feeling to the screen-door effect you observe. It’s not overly distracting, but you do know it’s there. 

Finally, the USB cable that comes with the display, to facilitate communications with the PC, is ridiculously short.  I would suggest they increase this to a 6-foot cable and not the 3 foot version provides.  It’s not possible to put the PC anywhere but immediately adjacent to the display.  This could be easily replaced, but for a $5,000 display, this seems like an unfortunate oversight. 

Finally, the angle of the display could be problematic depending upon your viewing environment.  Especially in a well-lit office environment with overhead lighting.  Effectively if you are not in a darkened room, anything bright on your ceiling is going to reflect into the screen. Much like what would happen if you positioned your regular desk monitor facing a window.  The specifications do indicate 100-1000 lux for ambient illumination is acceptable, but I would premise this as long as there is no light on your ceiling in direct reflected view of the display. 

As mentioned earlier there is an eye-tracking camera at the top of the display to assist with rendering the parallax effect.  While the display is meant only for one person, if you intend to sit side by side with a colleague or client while using the display, I found it could be problematic as it can’t tell who to track.  Even when using the display on my own, I sometimes found it had trouble finding me or would lose me in the middle of moving my head.  It was infrequent, but it did happen.  I also found that the cross-talk artifacts were significantly worse if I moved too quickly.  If I moved very slowly and deliberately, it was possible to almost reduce them altogether, although it was never removed entirely.  

The Sony SR Display is pretty damn impressive and a major leap forward for immersive display technology.  But like HMDs, there is a time and place for its application. The display feels complete and impressively usable, but still very much a first-generation product that is still trying to find its place.  

Like HMDs there is an obvious use case for immersion and getting a better sense of scale, but ultimately the power of this display will be in innovative and novel use cases where it serves a larger goal. Specifically, a use case where there is a cleverly authored real-time environment that does more than just display a model. 

I feel this display needs to be used in a way that enhances an existing real-time experience but is not used solely as a 3D gimmick.   As such, I do not expect this display will find its way onto the desk of every artist as a daily use monitor.  Maybe one day, but not yet.  

Within our industry, the most applicable use case will be within sales centers and in client presentations where there is a need to add another level of immersion to a real-time authored experience and to ensure the presentation experience is a memorable one.  If I was looking to create something truly unique in a sales center experience, I don’t think you could really go wrong here provided you understand its limitations. 

In a COVID-era where HMDs are not easily used or encouraged, this could be a nice alternative. It’s also a great alternative to provide some level of immersion where you might be dealing with stakeholders that are reluctant to put on an HMD, but you still want to provide a memorable and immersive experience.

While not an inexpensive display at $4999.99, if you understand its authoring limitations and want to create an immersive experience for your clients or sales suites, I do think this could be a very compelling alternative or addition to a VR HMD.  For more information and to purchase you can visit the Sony SRD site.

Unboxing Video

Promo Video

Mill Interview

Demo/Interview

Creators Interview