The search for 360° video VR... with two cameras
A few months ago we were assigned a project that was quite unusual. A wireless company needed to create a demo that showed the capabilities of shooting virtual reality content on a smartphone. Since the device technology featured only uses the smartphone’s front and rear facing cameras, we were tasked with finding a two-camera VR rig. Additionally, each of the cameras would need to shoot in 4K and the suggested VR frame rate of at least 60 frames per second.
We found out quickly that there are several prosumer rigs available on the market that were close to our needed specs. Although we knew the camera lenses were small and could potentially produce a low resolution image, we decided to test them.
Kodak PIXPRO SP360 4K Dual Pro pack
The dual pack includes two Kodak PIXPRO cameras and accessories that together shoot full 360° video, which is stitched after shooting. A12-megapixel sensor captures ultra high-def 4K video at 30 frames per second. It also has automatic features such as auto-focus, exposure, etc. Lack of manual control is not optimal for professional photos or video.
Samsung Gear 360
The Samsung Gear 360 uses one camera with two 180° spherical lenses. The camera sensor captures ultra high-def 4K video at 30 frames per second, which is then stitched with the provided Samsung software and exported as one file. Like the Kodak PIXPRO, It has all automatic features.
360 Fly 4K
The 360 Fly 4K camera has one spherical lens with a 240° field of view. This limits the captured video as it cuts off the bottom portion of the image. The resolution is 2880x2880 at 30 frames per second. While this is technically 4K, being that it is one video stream unstitched, it’s less overall resolution than the comparable models. It too has all automatic features, limiting the control for the shooter.
All the cameras’ lenses are very small, so small in fact that the image comes out in lower quality than just pixel ratio. More glass means better image!
Advertising at 4K is misleading. Yes, each camera does shoot 4K, but that includes the entire frame which contains more than 180 degrees of footage. When looking in 360 degree view, the viewing angle looks more like 480p at best. A better solution would be a rig that has more than two 4K cameras, such as the GoPro rig which can have 12 to 16 to 24 cameras. Of course, this would conflict with the needed spec of having to use only two cameras.
None of these cameras shot 60 frames per second. Only 30fps. We needed to find a solution…Better Quality, Higher frame rate. We needed a professional lens, a larger sensor, and an advanced body. The lens also needed to go beyond 190° in order to have a successful stitch.
While researching we found many wide angle lenses that were 180°-185°. This would not work though, as the overlayed image would be too small to stitch. We needed a lens that was in the 200° range. We researched more and found a candidate.
The Nikkor Lens
The Nikkor lens has a 220° field of view and mounts to large sensor cameras. When it was introduced in 1970 at the Photokina exhibition, it was the most extreme fisheye lens of all time - a 10lb glass dome which dwarfs the attached camera. Jeremy Gilbert, Group Marketing Manager at Nikon UK says, “The 6mm f2.8 lens is an incredibly rare lens that was initially designed for scientific and meteorological use. It represents the pinnacle in lens design, from a time when lenses had to be designed with a slide rule and individual ray diagrams. Lens production began in March 1972 and was only made available to special order.”
It looked huge, and promising. If we could test two of these in a back to back rig, could we get a better image than the prosumer rigs? This lens was extremely difficult to find, as it was produced many years ago in limited quantity. We found one lens at Panavision Los Angeles, and another which we had shipped from London. We set up a rental for a week later to test.
When the second lens was received we went to Panavision armed with two RED epic bodies and two Sony a7rII’s. Both cameras have large sensors, much larger than the prosumer cameras previously tested. After working on the configuration of the rig, we shot with both camera bodies. It shot an incredible image, though it did have problems for our use.
What we didn’t consider was the pixel ratio of the sensors. Both the RED and the Sony camera sensors are in a 16:9 widescreen ratio, not a 1:1 square ratio. The sensor captured the image, but the top and bottom were cut off. There was no way to reconfigure. We wouldn’t be able to get a full spherical image.
Another problem were the lenses themselves. Because they were produced so long ago, they weren't identical. This would make a stitch not work as the edges wouldn’t match.
The Nikkor lens wasn’t a viable solution. We found there wasn’t a solution for the 2-camera, full sensor, 4K, 60 frames per second assignment. We had to defer to one of the prosumer cameras and sacrifice quality and the preferred 60 frames per second.
We decided the best of the rest was the Kodak PIXPRO which we outfitted for the production. It was the most adaptable and had the best image of comparable cameras. We had to lose the 60 fps request, and limit to a 30 fps. We were then going to shift in the editing process to dump out 60 fps in order to fit specs. Not optimal, but for the demo it would work. It performed pretty well despite the automatic features on the camera.
There is no current professional solution for video VR using only two cameras. When doing VR, use a larger rig setup where you can utilize the high quality across an array of cameras, not just two, thus making the final output noticeably better. We love the Nikkor lens, it’s legendary, but we’ll have to utilize it when shooting stills or a non-VR project.