Shane Colton: Camera and Video

I've always used video as a way of documenting my projects, but I'm also interested in it from a technical standpoint. The process of capturing, storing, processing, editing, compressing, etc., and the tools that go along with it are pretty amazing. It can also be used as a tool for seeing things that you can't see otherwise. I've used high speed and synchronized/strobe imaging to troubleshoot vibrations, visualize motor controller timing details, watch roller chain dynamics, see Tesla coil spark evolution, and of course record go-kart drifting...

Camera Systems:

Wave High-Speed Camera [Product Page] [Post Archive] [Wavelet Compression]

Grasshopper 3 + Surface Pro 2 [Post Archive]

Camera and Video System Links:

[Freefly Systems] [I Heart Clare]

[FLIR Machine Vision (Formerly Point Grey)]

Wave Continuous High-Speed Video Camera

Status: Completed Summer 2020

Specifications:

Sensor: ams CMV12000

Resolution: 4096x3072x10b

4K Frame Rate: 300fps@4096x3072x10b (4:3), 422fps@4096x2160x10b (17:9)
2K Frame Rate: 1049fps@2048x1536x10b (4:3), 1461fps@2048x1080x10b (17:9)

Lens: E-mount
Image Processor: Xilinx Zynq Ultrascale+ SoC (Trenz Electronic TE0803)

Recorder: Internal NVMe SSD (continuous recording of 5:1 wavelet-compressed frames)

This is my first full camera build starting from the sensor. The CMV12000 has a lot of appeal for a custom build: The datasheet is readily available to mere mortals, it can be purchased directly from ams or through normal electronics distributors, and it's relatively easy to solder onto a board or find a ZIF socket for. It is expensive (~$2000), but in 2019 I managed to get some surplus stock of the monochrome version on eBay for $100ea to use for prototyping.

The second generation of this sensor is also mind-boggling fast, up to 3.8Gpx/s. To deal with that much data, I built a real-time wavelet compression engine that runs on a Zynq Ultrascale+ SoC. This compresses the 37.75Gb/s raw data by about 5:1, to a rate that can be written directly to an ordinary NVMe SSD via the SoC's PCIe transceivers. This allows self-contained continuous high-speed recording in a pretty small and low-power package.

The production version of the Wave camera, which includes an awesome locking E-mount, is available from Freefly Systems. The following are some images and videos from along the way.

Wave first prototype.

Sorting out first images from monochrome prototype.

Approaching final form.

Grasshopper 3 + Surface Pro 2

Status: Completed Summer 2016

Specifications:

Camera: FLIR Machine Vision (formerly Point Grey) Grasshopper3 (GS3-U3-23S6C-C)

Sensor: Sony IMX174

Resolution: 1920x1200x8-12b

Frame Rate: 160fps@1920x1200x8b, up to 1000fps at lower resolution

Lenses: Fujinon CF12.5HA-1 12.5mm f/1.4, Fujinon CF75HA-1 12.5mm f/1.8

Interface: USB 3.0 (370MB/s max)

Recorder/Display: Microsoft Surface Pro 2
Software: Point Grey FlyCap2, Modified Direct RAM Recorder

This is a machine vision/instrumentation camera that I got after searching around for regular video cameras and deciding that everything I ever take video of is in rapid motion and so I need a camera that has a global shutter. The Sony IMX174 is a strange thing - a global-shutter CMOS. Hopefully one of the first of many to come? Combined with the astonishing speed of USB3.0, it is capable of recording raw HD video at 160fps or lower resolutions at up to 1000fps. The global shutter speed can be as fast as 5us and it can synchronize its trigger to the outside world, essentially making it a visual oscilloscope.

Combine that with a large (1/1.2" sensor) with good low-light performance, and you get some really nice high-speed video for much lower cost than the next nearest thing that could shoot raw HD at 160fps. The downside is that it basically needs a PC tethered to it to record, and it chews through RAM faster than you can convert and save video, even onto an SSD. So the record time is limited at high frame rates.

I use a Microsoft Surface Pro 2 as a portable monitor and recorder for the camera, since it is a full-featured Windows tablet with USB 3.0 and enough RAM (8GB) to handle the task. It does have some trouble converting and saving video at full frame rate while also recording, so I've modified one of the Point Grey example programs to only buffer into RAM (deferring the convert/save step until later) for maximum frame rate. This gives about 20 seconds of record buffer at maximum frame rate.

I've used it to create some interesting high-speed and synchronized videos, as well as help troubleshoot on some projects. Below are some of the most fun results. The quality of the capture and processing software varies for these examples. See the post archive for the technical progression.