Colour-Science Digital Movie Enhancement Software (DiMES)

Introduction

Camcorder sales shows highest growth rates and more and more people produce their own video films.  Also normal digital cameras today allow to register short movies. Old cinema films have to be preserved from disintegration. They are digitized and are now distributed on DVD discs. Film is going digital and now more and more also image quality becomes an important issue. The time for nearly unrecognizable post stamp big digital movies is definitely over.

This is the time where the simple white balancing most of the video cameras are still using is not enough anymore. Like images from digital cameras also the frames of digital movies have to be enhanced. In fact there is no big difference in the enhancement of digital camera images to the enhancement of a digital movie. On the market there is a big number of video editing tools which can be used to convert video files between different formats or to cut and assemble films, but there is until now no video editing tool which allows an automatic enhancement of video streams.

In the photo industry image enhancement is a must to produce good prints. The DiMES project is now proposing such a movie enhancement tool for movie films, using Colour-Science i2e image enhancement and Color Management technology.

The DiMES Digital Movie Enhancement Software

The following graph shows the different processing steps of the DiMES software.

The enhancement steps

First we need to have a movie file to work on. If the goal is the restoration of old film material then the film must be first scanned. This can be done using a normal video camera, but if the archival of old films is the goal then a film should be scanned in a frame by frame process using a high resolution film scanner.

There are two different coding formats for movie film. The most used are the formats used to play films on computers or on the TV. These are formats like (mpeg, avi, divix, rm, wmv…). All these formats are time coded which means that not every frame is saved, but between frames only image differences from frame to frame are saved. But there are also frame based formats (mjpeg, pjpeg ..) which save every frame as a separate jpeg image.

So in a first step we have to convert image streams to a frame based movie format. Now everything becomes very similar to the enhancement of digital camera images. Colour-Science has much knowledge doing this and so we have ready to use software libraries for everything needed.

There are mainly two types of movie enhancement. If the movie comes from a video camera then it is just a normal enhancement. The color cast is removed, density and contrast is enhanced, details in the shadows and the highlights are made visible and memory colors like skin, sky and vegetation are corrected to more pleasant colors. These corrections are much more powerful then the simple  white balance and density corrections a normal video camera can today perform.

If the movie is a real film which has been digitized using a scanner, then a lot of times we have first to correct global color changes due to color fading. For this we have two possibilities. The first is a simple to use visual method. We just stop the film on some representative images and then we manually enlarge the gamut of the film by manually increasing color saturation round the color circle. We have 9 color saturation sliders for Red, Yellow, Green, Cyan, Blue and Magenta. It is also possible to make global changes to contrast , density and color balance. After this precorrection we run our normal i2e image enhancement over the film. This type of correction is enough good for most of the films. For high end restorations of films it is also possible to apply a global film precorrection based on ICC profiles. Here we use a profile editor to visually create a specially adapted icc input profile for the film. For this we use for example our editable sRGB profile (LUT form) and convert the images from sRGB(edited) to sRGB(standard).

In addition to density and color enhancement, also our local sharpening technology is very interesting for movie film. This technology is specially appropriate for the relatively small frame resolutions (compared to digital camera) from movie film. Sharpening together with our adaptive brightness and contrast enhancement will make the film look much more crisp and detailed.

In a big loop all frames of a movie are corrected. A normal movie will have 25x3’600=90’000 frames per hour. Amateur video films have normally quite a low resolution. The normal MPEG standard plays back at 352- by 240-pixel resolution at 30 frames. This means that the data of 30 frames or one second is about the same as one 3 Megapixel digital camera file. So on a fast computer processing speed can  be enough for real time processing. For higher resolution video streams (HDTV) the processing speed will be slower.

Applications

Movie film enhancement is a big market with a large amateur segment. There are a lot of possible applications.

An interesting application would for example also be to implement the core i2e image enhancement library directly on the video camera. The i2e image enhancement library provides all necessary routines needed to process the raw image of a CCD or CMOS image sensor and produce a perfectly corrected 8 bit image. The main enhancement steps can be implemented easily on the internal processor of the imaging device.

The routines are written in pure ANSI C/C++ and can be implemented on nearly any platform.

An other interesting application would be to implement our image enhancement and color management directly on video projectors and enhance video projections in real time. This would be also possible as a software solution where we enhance the video stream in real time on the computer before we display it using a projector or the monitor.