The low cost high frame rate camera
Two years ago I created the fps1000 low cost high frame rate camera. At first I didn’t know much about photography or cameras, but I was fascinated by photographs which froze motion in time and I wanted to learn more. It was also a technical challenge; as a long time inventor and hardware design engineer I liked the challenge of being able to move huge amounts of data around at high speed. Traditionally, high frame rate cameras have been very expensive and bulky – I wanted to achieve this in a low cost, small hand held device.
The early prototype certainly achieved the basic design goals. Have a look at the original fps1000 kickstarter to see the full story. But it had many shortcomings – it needed an external power supply, the LCD screen was very small and the case was flimsy.
With feedback and help from the kickstarter community I addressed these shortcomings and produced the final fps1000. This was a steep learning curve taking me into areas that I had never experienced before such as product design. It was tough and took a lot longer than I anticipated. I went through several different attempts at case design until I had learned enough to come up with a design that was attractive, robust and easily manufacturable at a sensible cost. I decided not to cut corners and aim for a product that I was going to be proud of. I think I have achieved this and the feedback from customers has been how well engineered the product is.
Meanwhile the internal electronics went through a complete re-design. The essence of the design, the sensor and memory board was very little changed from the original prototype. But I added a second board – the motherboard – which housed a new, high speed DSP processor, a battery holder for internal batteries to make it self-contained, trigger inputs and outputs, a USB 3 port and a new 5 inch touch LCD display.
The camera software was completely re-written for the new hardware. All operations are available via the LCD touch screen, allowing you to record playback and save image sequences to the built-in micro SD card.
The new camera is significantly superior in appearance and functionality. This was completely funded by me without any extra charge to original backers.
Building on the success of the new design, the fps1000 HD utilises a new image sensor that is significantly better than the original. The rest of the camera architecture stays the same so that the new camera can take advantage of the motherboard, case and software of the original design. These are all compatible with the new sensor board saving a huge amount of new development time. On the new sensor board the memory has been doubled to 128 GBytes to cater for the faster data rates, retaining the one full minute of record time.
The new fps1000HD sensor compared to the original fps1000 sensor:
The case is the same as the current fps1000 except it has added cooling to cater for the extra power of the new sensor and larger memory.
The fps1000 differs from many high frame rate cameras. Firstly it is completely self-contained. Many high speed cameras need to be operated from a computer and have no built-in screen. The fps1000’s built-in LCD screen allows you to easily frame the scene and focus as well as play back clips once taken.
Secondly, the fps1000 has an unprecedented one full minute of record time at full frame rate. Most high speed cameras can only record a few seconds of video making it difficult to capture the required action.
Finally, All clips are recorded in raw format – there is no compression in the camera which means that all recorded image data is available in post-process to allow colour grading. The frame rate is also constant with no frame dropping or resolution compression. The raw data is saved in an industry standard DNG digital negative format (an Adobe open standard). This can be directly read into many post processing packages such as Final Cut Pro and Lightroom.
The combination of integrated LCD screen, raw data, memory capacity and battery operation make the fps1000 unique in this price range.
So what’s all the fuss about recording raw video? Most video cameras produce encoded video files. The process of encoding compresses the image to reduce the file size so decisions have to be made about what data to discard. This means that some of the original information about lighting or colour balance is lost. Also the pixel depth is reduced – typically down to 8 bits – loosing more colour information and dynamic range. It is impossible to reconstruct this missing data once it is lost.
With the fps1000’s raw files no information is lost. Each frame retains the full 12 bit colour information and full resolution so it is much easier to post process: to colour grade, adjust contrast and brightness or to add effects. Once a video is compressed in camera, then some of the original information is lost making it difficult to work with after.
Another great advantage of raw video is that it can be upscaled to higher resolution during the final encoding process so recording at higher frame rates and lower resolution and then upscaling to 720 is possible with very little loss in image quality.
The final image quality is a combination of many factors – resolution is one of them, another is pixel depth. But the encoding method has possibly the biggest effect on image quality. A poorly encoded high resolution image can look worse than a well encoded low resolution signal. The choice of encoding method with cameras that produce encoded video files is usually very limited. It will be governed by the available algorithm on the chips in the camera used to encode the signal. One factor governing which algorithm to use is the power required in the chips to do this. The best encoding methods will take a lot more power and are often not used. With raw files, however, you can choose whatever encoding method you like and you will have a much more powerful computer or graphics processing unit to do this. So while most cameras will encode to 4-2-2 or even 4-2-0 you can choose to encode to a much higher standard such as 4-4-4.
Native resolution is 720p HD. At this resolution the fps1000HD gives you a continuous 1,000 fps at 12 bit pixel depth. No frame skipping or compression.
For even higher frame rate you can drop the resolution down to 640 x 360 and achieve 4,000 fps! This is also continuous recording rate in full raw quality at 12 bits per pixel.
To achieve this frame rate the camera needs to be 4 times as sensitive to light as the exposure time is one quarter of the exposure at 1,000 frames per second. This is achieved in the fps1000HD by a method called binning. This technique combines 4 pixels into one, reducing the resolution down to 640 x 360 and at the same time producing “super pixels” 4 times bigger and hence 4 times as sensitive to light, thus maintaining the required light sensitivity. This applies to both colour and monochrome camera variants (binning is often only achievable with monochrome sensors).
This is a powerful option for the highest frame rate requirements and is available through a control on the touch screen. The sacrifice for the increased frame rate is a reduced resolution. However, this is not as bad as it seems. Thanks to the raw recording capability, 640 x 360 is actually much higher quality than many full HD encoded signals, so the apparent image quality is much higher than the resolution may indicate. See the comparison between the two bee videos below. The video of the bee at 1,000 fps is very impressive but the wings are still a bit of a blur. Speeding things up by a factor of four shows the majestic rotating wing action in detail.
The fps1000 family uses a global shutter sensor. This means that all pixels in the entire image for each frame are captured simultaneously. Many sensors use a rolling shutter meaning that lines are snapshotted sequentially causing the so-called “jello” effect on fast moving images. Only a global shutter is suitable for distortion-free high frame rate photography.
Using the touch screen interface it is simple to change the frame rate. The fps1000HD provides further fine control over image capture by adjusting the exposure time for each frame. The exposure time is expressed as a percentage of the total frame time and can be easy changed from 100% (i.e. 1/frame rate) down to 1%. This is a very powerful facility. Typically the exposure time is set st 50% – this means that for half of the frame time the image is being captured. For a 1,000 fame per second capture rate the exposure time is 1/2000 of a second.
A shorter exposure time provides for less motion blur but requires better lighting as the sensor is exposed to light for a shorter time. A longer exposure requires less lighting but increases motion blur. An exposure time of only 10% will provide very sharp images with minimal motion blur, equivalent to a 10,000 frame per second exposure time even with 1,000 frames per second frame rate.
The variable exposure time also provides a form of aperture control – reducing the exposure time is equivalent to closing down the aperture as far as light levels are concerned.
You can quickly experiment with frame rate and exposure time by taking sample shots using different setting combinations.
The video below shows the effect of reducing exposure time. The entire video is a close-up of a water jet shot at 1280 x 720 and 1,000 fps. The video is split into 4 sections with successively shorter exposure times. At 100% exposure time (1 millisecond) you can clearly see motion blur by the elongated water droplets. At 50% exposure (500 microseconds) the droplets are slightly less elongated. At 20% exposure (200 microseconds) the droplets are almost distinguishable as balls and at 10% exposure (just 100 microseconds) the droplets are clearly identifiable with almost no signs of motion blur.
Of Course as exposure is reduced so too is the amount of light entering the sensor so there needs to be sufficient lighting available.
The fps1000HD is small, light and completely self-contained. It can run from 2 Li-ion internal batteries for 1-2 hours of typical use. Many high frame rate cameras require a host PC to preview the scene and to transfer data in real time for storage on the PC. The fps1000HD has a real-time preview feature using the built in LCD display. This allows you to frame and focus your target for recording. Real time histograms and focus peaking allow light levels and focus to be adjusted. There is no image lag which could be introduced over a relatively low bandwidth PC link.
This portability allows the camera to be used where a moving point of view is required. This is becoming very popular in video work where the camera can be mounted on a rig or a drone to produce moving point of view videos. The duck video below uses this technique. The camera was panned quickly across the scene as the action was occurring to produce interesting freeze effects.
The fps1000HD native lens mount is C Mount. This is a very compact mount and there is a wide range of low cost C Mount lenses readily available. The lens provided is a 35mm focal length f1.7 lens with manual aperture and focus adjustments. An extension ring is supplied for macro work. With the addition of a simple lens adapter most lenses can be supported. All of the videos produced here were either using the standard supplied lens or Canon prime lenses via a low cost adapter.
Workflow
The videos are captured as a sequence of files. Each file represents one captured frame, so at 1,000 fps there are 1,000 individual image files per second. Each frame is captured as a 12 bit image and contains the raw data for the image. The file format is Adobe DNG (digital negative). These files can be read by a wide range of applications. The name of each file is the frame number (frames are numbered from 0 to 65,000) with a .DNG extension.
A typical workflow involves saving the images of a chosen clip to the built in micro SD card, copying the files from the saved clip onto a computer and then loading them into your chosen video compilation app. Final Cut Pro is a popular choice; this can read the DNG files directly and then compile a video from them. An alternative is to use Adobe’s Premiere Pro, however this cannot read the DNG files directly so they need to be converted to JPG or PNG files first with a program like Adobe’s Lightroom or the freely available RawTherapee. All of the videos here are produced using RawTherapee and Premiere Pro.
There are 2 versions of the fps1000HD. The main product has 128 GBytes of memory, enough for 60 seconds of full speed recording (more at lower frame rates). Maximum frame rate is 1,000fps at 1280×720. There are early bird savings in several tiers for this version.
A lower cost version has 64 GBytes of memory and slightly reduced performance at 750 fps at 1280×720. All other features are the same as the full version.
Each model is available in monochrome or colour versions. You will be able to specify which version you want after the campaign is closed.
The fps1000HD development is complete. There are 3 main components: the motherboard, the sensor board and the case. The motherboard and the case are the same as those for the original fps1000 and are well proven and in production. The case is identical to the current production case except for a modified front panel with cooling fans added. A number of prototypes of this have been produced and a production batch is under way. The sensor board is on its third iteration and is bug free. A number of these have been produced already and are working correctly. Several prototype cameras have been continuously used for several months and are performing well. The software is largely compatible with the current production versions and the required modifications for the new sensor board have been made and are working well.
The sample videos shown here were all taken with the fps1000HD as supplied. The lenses used were a mixture with about half using the standard lens supplied, and the rest using prime Canon lenses with a low cost adapter. The standard lens with the extension ring supplied was used on the butterfly and bee videos. There were various colour gradings used during post processing which explains some of the colour variations especially noticeable on the butterflies. This was just a case of me experimenting. I’m far from experienced at grading but I am getting the hang of it gradually. There are no other special effects applied like sharpening – apart from the colour grading all videos are as they were produced from the camera. All videos except the ducks were taken with a handheld camera in normal daylight. The ducks were taken with a tripod-mounted camera so the moving point of view could be smoothly applied.
Many thanks to Olivier Behzadi at Sassy Films and Sam Edwards at Idoru Productions for your time and contributions to this campaign. Thanks also to Jonathan Rose and team at Motionfish for the kickstarter video.
The fps1000 HD is a complete, tested design. Many prototypes have been made and are fully functional. Most of the hardware and software have been used for some time in the existing fps1000 product and are well proven. The key to success of the fps1000 HD is the ability to purchase memory components in sufficient quantity to make the cost price of the camera affordable. The memory lead time is the only risk to the success of the project but this has been built into the delivery time estimates.