Taking Control of DirectShow® with VQLab

Introduction

DirectShow® is a multimedia architecture developed by Microsoft, which divides the processing of multimedia tasks such as video playback or video effects into a set of steps known as filters. Filters have a number of input and output pins which connect them together. The generic design of the connection mechanism means that filters can be connected in many different ways to achieve different tasks, and developers can add their own effects or other filters at any stage in the graph. DirectShow® filter graphs are widely used in video playback (in which the filters will provide steps such as file parsing, video and audio demultiplexing, decompressing or rendering) as well as being used for video and audio recording and editing.

DirectShow® connects two filters by using the Intelligent Connect mechanism, which adds the necessary intermediate filters so that the connection can be done, by using several internal algorithms.

VQLab uses the DirectShow® framework to perform several tasks like reading, decoding, demultiplexing video streams and performing video quality assessments.

The DirectShow® framework is a dynamic environment, which is changed, usually without any notice, by different products, like codec packs. These changes mean adding or removing new filters. Problems appear when new filters for tasks like decoding or demultiplexing are added, because they can be used instead of previously installed filters.

To be able to maintain the reliability and reproducibility of the video quality tests, VQLab offers the possibility to:

• use the reliable set of VQLab filters (set High Priority for VQLab Filters option)
• manually edit the filter chain used for every video file
• save DirectShow® graphs as vqg files for latter use
• associate DirectShow® graphs with video file extensions.

High Priority for VQLab Filters

VQLab is using the industry-leading MainConcept video codecs to support the following video formats: MPEG-1, MPEG-2, MPEG-4, AVC/H.264, VC1 and DV.

When setting High Priority for VQLab filters option, located under File menu, VQLab will instruct the DirectShow environment to always use the reliable set of VQLab/MainConcept filters. To use the default DirectShow® Intelligent Connect, simply uncheck High Priority for VQLab Filters option.

Manually Edit the Filter Chain

By using the powerful built-in graph editor of VQLab, you can add new filters, remove unwanted ones or change the properties of the currently used filters.

As you can see in the picture above, an usual filter chain will contain a source filter, a demultiplexer, a decoder and a VQLab Metric Filter (VQLab PSNR Filter in this case).

When a video file is added as reference or processed video, VQLab connects the source filter that loaded the video file to a VQLab Metric Filter, by using DirectShow®’s Intelligent Connect mechanism, which creates the filter chain by using several internal algorithms. The resulting filter chain can be manually edited by changing existing filters and/or adding new ones. This is very important when you want to have full control of your DirectShow® filter chain for performing more complex tasks (e.g.: resizing on the fly) or when you want to remove filters which proved unreliable.

DirectShow® Graph as vqg File

You can save a finalized graph as a .vqg file and store it on your hard-drive. A finalized graph is a graph with the last filter a VQLab Metric Filter. The save/load vqg file options are available in VQLab’s built-in graph editor, under Graph menu, Save->To File… for saving a graph as vqg file and Load->From File… for loading the graph from a vqg file.

A vqg file contains information about how the video file, loaded through a source filter, connected to the VQLab Metric Filter, including the filters needed to make the connection and their properties. This way, when using VQLab afterwards, you can load a previously saved vqg file for one or more video files, using the same filters, including their properties.

Remarks: Please keep in mind that a vqg file can be successfully loaded for a video file only if it has the same type (encoding standard (MPEG-1, MPEG-2, H.264/AVC, etc), container (AVI, MOV, MKV, MP4, etc)) as the file from which the vqg file was saved. It is considered good practice to give suggestive names to the vqg files, so that you know for what type of video files the vqg file corresponds to.

Associate DirectShow® Graphs with Video File Extensions

Associating DirectShow® graphs with file extensions has a great impact on the time needed for a video file to be added to VQLab. When adding a video file, VQLab first checks to see if the file extension has an associated graph, if not, it tries to connect the source filter used to load the video file to a VQLab Metric Filter by using Intelligent Connect, which is a time consuming process. On the other hand, loading a video file that already has a DirectShow® graph associated with its extension is a very fast operation.

Associating DirectShow® graphs with file extensions assures you that video files of a certain type will always be processed in the same way by VQLab, regardless of the DirectShow® environment changes due to codec pack installation or other applications installations.

Associating DirectShow® graphs with file extensions is a very useful option, enabling you to create reliable test environments, but it must be used properly to obtain the best results. Below there is a list of advice that should be taken into consideration when associating vqg files with an extension.

1. Video file extensions should define the type of encoder used, and not the type of container.

Usually, video file extensions define what container was used for the file (avi, mpg, mp4, mkv etc). The quality of a video file is dependent on the encoder used rather than the container. A container can support various types of encoding, for example mpg can support MPEG-1, MPEG-2 and H.264/AVC. If, for example, you associate .mpg extension with a graph that can decode MPEG-2 streams, it is highly probable you will not be able to use that graph for an H.264 stream. Extensions like: mv2 (for mpeg-2), h264 (for H.264/AVC) are recommended instead of mpg general extension that does not differentiate between types of encoding.

2. Video file extensions should define the type of stream used, if several types are supported

De-multiplexers support, in general, both transport streams and program streams. If you create a graph that can decode program streams, and associate it with mpg extension, loading a mpg file that contains a transport stream will fail. Extensions like mts (mpeg transport stream) or mps (mpeg program stream) are recommended in this situation, to avoid confusing the de-multiplexer used.

3. Color Formats

VQLab Metric Filters accept connections in a various range of color formats: AYUV, UYVY, YV12, Y41P, RGB24 and RGB565. A filter that connects directly with a VQLab Metric Filter sends data in one of the color formats accepted by VQLab Metric Filter, which is saved as the graph color format. When saving the graph as vqg file, or associating it with a file extension, the color format of the graph will also be saved. If you save a graph to vqg file in UYVY, and then try to load it for a video file that has the same container and encoding, but its color format is RGB24, it is possible that this operation will fail. In this situation you can add in the graph a MainConcept (VQLab) Color Space Converter filter before the Video Metric Filter and then save the DirectShow® graph as vqg file, or associate it with an extension. This will make the graph suitable for a wider range of color formats, including our example (RGB24).