Chimera 2: Putting An Emphasis On Imaging
Nvidia was clearly bullish on imaging when it introduced Tegra 4. Given the company’s strength in graphics, it made sense that it’d extend expertise to photography and video as well. Disappointingly, we waited almost an entire year before the first manifestation of the company’s Computational Photography Engine surfaced, enabling always-on HDR and video stabilization. Then again, given the dearth of devices sporting Tegra 4, a modest software ecosystem isn’t altogether surprising. Should Tegra K1 enjoy more rapid pick-up, we’d hope to see hardware manufacturers doing more with the SoC’s imaging capabilities.
The potential balloons with Tegra K1. Last generation, Tegra 4’s ISP was rated for up to 350 megapixels/s of throughput. This time around, dual ISPs offer 600 megapixels each (two 20 MP streams at 30 Hz). Having a pair of ISPs make it possible to gather images from one source as information is processed from another, either from two cameras or from a camera and memory. A crossbar fabric connects both ISPs to memory, where they’re able to communicate with the CPU and GPU.
There’s a lot of future-looking enablement going on—support for up to 4096 focus points, 14-bit input, 100 MP sensors, interoperability with general-purpose compute, and quality-enhancing capabilities to minimize noise, correct bad pixels, and downscale are all at least possible. Some of the features are in place simply to improve quality, while others may pave the way for new imaging applications.
For instance, allowing more than 4000 focus points seems like overkill. Nvidia counters that this is useful for tracking multiple objects, and indeed we’ve seen demos of moving content where the camera detects and tracks the subject, maintaining focus on it, despite the rest of the scene changing. In the same vein, leveraging the graphics engine’s newfound compute capability, effects can be applied to the ISP’s contents in real-time. Deep images, where each pixel can store any number of samples per channel (instead of just one), become viable. So does creating panoramic shots by “painting” with a camera sensor.
Powerful ISPs backed by a capable GPU open the door to new forms of artistic expression, and Nvidia’s team clearly has the vision to drive innovation forward in that space. As mentioned, though, demonstrations of what computational photography can achieve have been somewhat slow to emerge in production. By taking the reins and introducing its own Tegra Note 7, Nvidia put itself in a position to roll out new software features. The first OTA update appeared mere weeks ago. Hopefully, that cadence speeds up with Tegra K1—we don’t want to wait another year before the company’s most recent demos become tangible.
When Nvidia introduced Chimera last year, it presented a slide very similar to this one, except that there was only one ISP and the GPU wasn’t Kepler-based.
