Colour global-shutter image sensors for multi-camera products

01

Synchronising the cameras on one machine

A humanoid uses more than one camera. The head watches the person, the chest watches the floor, and the wrists watch the grasp. If those exposures disagree by a millisecond, the hand stored from the wrist camera is not the hand stored from the head camera.

Today that agreement is a sync box and a harness. GS2.3 carries the clock. The sensors elect a master, measure the cable, and stamp every frame. The target is 100 nanoseconds on a board, and 250 nanoseconds across five metres of cable.

Four to eight cameras is the usual body. The one at the hand is about 30 cm from the object. The one on the head is often a metre and a half back, with a window behind the person that can be on the order of a thousand times brighter than the room. At 1 m and 1 m/s, on a 90° field, one pixel of blur wants about a millisecond of exposure. Closer than that, it has to be shorter still.

02

High dynamic range in a single exposure

On the global-shutter sensors used in robotics, an HDR mode is often a second exposure, or it keeps roughly half the frames. The bright window is then recorded later than the dark room. A hand that moved in between is stored twice.

GS2.3 reads the shadow and the highlight from one global shutter. Dim light stays on a high-gain path. Light past the photodiode spills into a capacitor in that same exposure and stays linear. Both paths are digitised every frame, so the target stays 90 frames a second with the range on.

At 90 fps the practical exposure is about 7.5 to 9 milliseconds, not the full 11. Overflow charge from the frame being read has to clear before the next exposure reuses the node. That costs under 2 dB. It is how the hand stays one hand.

03

Colour when a white highlight clips

On a white surface the green channel usually reaches full scale before red and blue. If green is clipped and the other two are still rising, the recorded colour is magenta. A white lamp in the room becomes a pink lamp in the file, and a model trained on that file treats the pink as real.

GS2.3 reads both the high-gain path and the overflow path for every pixel, every frame. A colour quad switches together. Channels reach the limit together, and a clipped pixel is flagged instead of being filled in. Path gain is calibrated per pixel from −20 °C to 85 °C. The gate is 1% error, 0.5% on green.

The range we design to is the span where luminance signal-to-noise stays at least 10 and no channel is clipped. Tier 1 is 80 dB in one exposure. A bigger overflow store can print more decibels and still fail this test.

05

Worn on a person: headset, cap, hard hat.

A stereo pair at roughly the spacing of two eyes is how a person demonstrates a task, drives a robot from their own view, or runs visual-inertial odometry while walking. The head is never still. A rolling shutter shears the doorway. Two cameras that do not share an exposure centre do not see one stereo pair.

The same pair sits in the brim of a cap when the headset is too much mass, and under the brim of a hard hat on a site or in a warehouse where the operator already wears a helmet. In each case the sensor job is the same: global shutter, matched colour, and a trigger that is not a box on the belt.

06

Cameras on the head, and a ring around the cell.

A head rig carries a forward pair and cameras out to the sides, with the computer and the battery in a pack on the back. The head only has to carry glass and shutters. The cable down the back is long, which is where a sync box becomes a nuisance and where cable delay has to be measured, not guessed.

Nine cameras is a ring around a work cell: a person and a robot, a bench, tools. The product is one time base and one calibration for the ring. Nine independent shutters are nine different moments. GS2.3’s job here is SyncLink — master, delay, timestamp — and colour that does not invent the lamp over the bench.

07

A gripper that teaches, and a wrist that repeats it.

One way to show a robot a grasp is to do the grasp yourself, with a camera looking along the fingers at the object. The camera sits on the spine of a pistol grip, the jaws in the foreground, the object a few centimetres further. Metal, plastic and a lamp are in the same frame. The hand is moving.

The robot then wears the camera on the wrist, about 30 cm out, and tries to repeat what the person did. If the teaching camera and the wrist camera do not share a shutter type, a colour response and a notion of time, the demonstration does not transfer. A single camera is enough when depth is computed elsewhere. A pair is there when the grasp itself needs stereo.

08

A drone between cloud and field.

The forward camera has sky in the top of the frame and ground in the bottom. The aircraft is what is moving, so a rolling shutter shears the map, and a second exposure for the clouds shears it again. Navigation wants the frame rate. Inspection wants the colour of the crop or the roof, measured in visible light.

Near-infrared does not supply that colour. The filter dyes are transparent there, so any colour in that band is invented. A synchronised white flash is for close work: in the model, about 4.5 W average at one metre for a clean colour image in one millisecond. It does not light a field from altitude.

09

Recording a demonstration around a table

Cameras on stands, in a ring, record a person folding cloth. The recording is usable only when every camera saw the cloth leave the hand at the same time. A camera that is late has not taken another view of the same event. It has taken a different event.

The clock on the floor, and the person whose job is the clock, are what this costs today. A lamp over the table is the other cost: when one channel clips and the file invents the rest, the recording contains a colour the room never had. The flag on a GS2.3 pixel is how that sample gets dropped instead of learned.