Global Shutter, CSI Cameras in Stereo – What that Means
The vision systems on Spiri drones are based on global shutter, CSI cameras in stereo. This essay explains our choices and what they mean for you. We’ll begin with definitions and comparisons, and end with a discussion of how our cameras are used.
Global Shutter Vs. Rolling Shutter
A camera sensor with a global shutter scans the entire sensor frame at once. With a rolling shutter it scans one line of pixels at a time, typically from top of the sensor frame to bottom.
The problem with rolling shutter is that when a scene is moving quickly, the recorded images are distorted because the first line of the frame is recorded before the last line of the frame. If a camera drives alongside a building, the building will look slanted. If a camera records a propeller, the propeller will look curved. The effect is made worse the higher the resolution of the rolling shutter camera because it takes longer and longer to scan the frame. Global shutter circuitry solves this problem.
CSI Bus Vs. USB Interface
CSI (camera serial interface) is a direct connection to the computer’s image processing systems, while USB (universal serial bus) puts a layer of hardware abstraction between the camera and the computer. The reason for running cameras through USB is that typically they are easier to integrate; the drivers ordinarily work out of the box. But for the convenience you pay severely in performance.
Bandwidth is the first problem when running a camera through USB. Even where the theoretical bandwidth for USB3 should be sufficient for video, there are still processing and memory bottlenecks that throttle the bandwidth. The second problem is graphics processing performance. A USB camera feed will go straight to the CPU and can easily overload it.
The CSI bus, on the other hand, opens up the maximum bandwidth for video, and routes the camera feed into the hardware vision pipeline (instead of using DRAM), which reduces CPU load. Not only does this lead to much more efficient handling of the video stream at the hardware level, it opens up a plethora of low level software controls to help process images.
Spiri uses the CSI bus for its two global shutter cameras.
Stereo Vs Monocular Vision
Like our own two eyes, the primary advantage of stereo cameras is depth perception. Spiri synchronizes its cameras by having one trigger the other, similar to our own vision with one eye leading, or “dominant”. The difference between the two images is perceived as depth, and tasks such as obstacle avoidance and 3D mapping are more accurate and faster than with monocular (one eye/camera) vision.
Bringing it Together on Spiri
Spiri uses its vision system to assist in navigation with a process called SLAM (simultaneous location and mapping). Spiri creates a 3D map of its surroundings, and uses the motion of features in that map as an indicator for its own motion, approximating how animals use vision to navigate. With our global shutter cameras and CSI interface, Spiri can rely on its “eyes” to fly, with bandwidth and processing cycles to spare.