idea/programming: Uwe Freese, Soenke Luetjohann
doc: Uwe Freese
This filter deinterlaces an interlaced video. One field is interpolated, the other is returned unchanged. Interpolation is done pixelwise in the time dimension
I made it to deinterlace some strange material from DVB that shows irregular movements when not deinterlacing or deinterlacing with usual deinterlacers.
Interlaced video consists of frames with two fields, a top field (all odd lines) and a bottom field (all even lines, counting up from one). The fields show the filmed scene at different times, because the camera that was used made twice as much pictures per second as the framerate is.
When viewing such a video, the player (generally a TV set) shows first the one, then the other field of a frame and then goes to the next frame which again consists of two fields.
Processing interlaced video material with VirtualDub (and a lot of other programs) means to ignore that the fields are shot at different times and to show both fields at the same time as one image. That results into interlace lines in moving scenes.
To correct this, there are the deinterlacers, programs (or "filters") that generally blend each two lines in a frame. But all the interlacers I found only make the interlace lines go away by blurring the areas that show motion or even the whole frame. They don't use the fact that the one field shows a picture made a little earlier than the other field. It doesn't matter in general, but with my corrupted material, it does.
I have material that was produced in any way, but was processed with filters before broadcasting as interlaced video (e.g. to change the framerate).
The broadcasted video has frames that show interlace lines and other frames that don't show them (both in moving scenes). The sequence of this types of frames has no fixed order and seems to be randomly.
When processing this kind of video with a deinterlacer that throws one field away in the areas where interlace lines can be seen, your resulting video makes irregular movements. That's because input frames with interlace-lines are deinterlaced to frames that show the scene at the earlier time or the later time (depending on which field is thrown away). And frames that don't show interlace-lines are returned "unchanged" (regarding the time). Even processing only one field doesn't result in smooth movements.
Exactly this is what happened to me with the StarTrek TNG series broadcasted over DVB and was the reason to build this filter.
Frame 01: No interlace lines.
Frame 02: Weak interlace lines.
Frame 03: No interlace lines.
Frame 04: Weak interlace lines.
Frame 05: No interlace lines.
Frame 06: Weak interlace lines.
Frame 07: No interlace lines.
Frame 08: Heavy interlace lines.
Frame 09: No interlace lines.
Frame 10: Heavy interlace lines.
Frame 11: Weak interlace lines.
Frame 12: Heavy interlace lines.
Frame 13: Heavy interlace lines.
Frame 14: Heavy interlace lines.
Frame 15: Heavy interlace lines.
The idea of the time based deinterlacer (TBDI) is to return one field unchanged and the other field as it could be at the corresponding time by calculating the average value pixel by pixel in the time dimension.
When using TBDI, a really important thing to know is which field is the earlier one. We call a video where the top field of a frame should be showed first a top field first video, a video where the bottom has to be shown first a bottom field first video.
You have a video where the top field should be shown first, then the bottom, but without this deinterlacer the fields are shown at the same time. That means that the bottom field is too early.
To correct this, you have to delay the bottom field. The easy way is to let the filter return the top field unchanged and an average of the current bottom field and the bottom field of the frame before. That's called delay bottom - interpolate bottom.
The second way to correct this is to return an average of the current and the last top field and the last bottom field unchanged. It should produce a comparable result, but there could be a case where it's better using this solution.
Because also the field that's not interpolated is kept in a buffer, this method is a little bit slower than the first one. There's also a video delay of half the time that's between each frame (20ms for a 25 fps video). This method is called delay bottom - interpolate top.
You have a video where the bottom field should be shown first, then the top, but without this deinterlacer the fields are shown at the same time. That means that the top field is too early.
To correct this, you have to delay the top field. The easy way is to let the filter return the bottom field unchanged and an average of the current top field and the top field of the frame before. That's called delay top - interpolate top.
Again, there's a second method, a little bit slower, with a video delay. It's called delay top - interpolate bottom.
In "correctly produced" interlaced material, both fields of a frame show the same scene, but with a time difference. Especially when the scene changes, both fields show either the old scene or the new one. When using TBDI without a scene detection, it will result in a "fading effect" in scene-changes. For a better result, TBDI detects scene changes and returns a complete image of the new scene there.
An adjustable amount (testpixel distance) of pixels of each field of the video is analyzed for detecting scene changes.
It is calculated how much these pixels changed their color values comparing to the pixels at the same position in the previous frame, and then considered as changed or not, depending on the color difference threshold.
At the end of analyzing all test pixels, it's calculated how many of them changed. Depending on the pixelcount threshold, the field is considered as changed or not.
Depending on how you set the detect scene change only when both fields differ option, the whole frame is then considered as changed, if one field / both fields changed. When this happens, the field with the greater amount of changed pixels is returned unchanged from the filter input and the other field is interpolated spatial for a sharp picture.
For a better performance, analyzing the frame for scene changes and the usual time based deinterlacing is implemented as one program loop.
The minimal scene change distance option is useful for the scene detection for not to "fire" on every frame in a scene with great changes like an explosion. In such cases, the passage is processed in usual TBDI manner, not with a scene change correction.
scene detection OFF: When this mode is set, no scene detection is used, and the filter runs completely without analyzing scene changes.
scene detection ON: When this mode is set, scene detection is enabled.
show bargraph: With this mode you can view unchanged pictures with a bargraph that shows how much the actual picture changed. The bargraph even shows if the filter would consider the frame as a scene change (green bar) or if it wouldn't because of the minimal scene change distance (red bar).
show scene changes: In this second test mode, the TBDI returns a half-green frame at a scene change and a red one if it detected a scene change that wouldn't be used because of the minimal scene change distance.
testpixel distance: You can set how many pixels should be analyzed with this option. Every 1000th pixel means about 200 pixels for every field of a full resolution PAL or NTSC video and is safe enough for detecting scene changes correctly. The more pixels the filter analyzes, the slower it is, of course.
color difference threshold: If the color difference between a pixel and the same pixel in the last frame is greater than the color difference threshold, it is considered as changed.
pixelcount threshold: How many pixels have to be considered as changed to detect a field as changed can be set with this value.
minimal scene change distance: A detected scene change is only re-interpolated (spatial as described above) when the last scene change is more or just as much frames before than this value. The counter for the last scene change is reset not only by a scene change that's re-interpolated, but with every detected scene change. That means when 100 frames (e.g. an explosion scene) are detected as changed, only the first picture is re-interpolated.
detect scene change only when both fields differ: The analysis is field based, that means that it's calculated if the top field and if the bottom field show a scene change.
When the option is disabled, a frame is detected as scene change when at least one field was detected as scene change. When this option is enabled, a frame is only detected as scene change if both fields were. That means that TBDI detects less scene changes.
There could be a case where it's useful to enable this option, but I recommend to disable it in daily usage. In both cases, the filter will spatially interpolate the filed with less changed pixels.
VirtualDub is (c) by Avery Lee and can be found at the VirtualDub homepage.