DirectShow, Media Foundation, DXVA, what?

Question

I have been instructed to review an application that uses DirectShow to render video. The old application works fine, but it starts showing age.

Since I have a gaming background, I decided that I should just go for it as such. Having made a bunch of prototypes using different technologies, I was sure that I could pull out the video rendering into the texture and use DirectX to fulfill all the requests of all clients.

However, I am now in the fact that I need to choose a technique, and I could not lose more ...

• Directshow
• Media fund
  • Media Session (<Windows 8)
  • Media Engine (> Windows 8)
• Dxva1
• Dxva2
• DXVA-HD

DirectShow is obviously the oldest, but it still works very well, also mainly due to the huge amount of codecs available.

The Media Foundation was introduced with Windows Vista and had something called a Media Session right up to Windows 8, after which they introduced the Media Engine. If I correctly codecs are called MFT? Are there really sites that have features like free-codecs.com?

In addition to these two methods, there is DXVA, which allows you to disable video decoding on the GPU. Currently, I am faced with three different types. DXVA1 is an implementation that has existed since XP, DXVA2 is a new API. But what is a DXVA-HD? Is it newer than DXVA2, or is it an equivalent name?

I really want to go to the Media Foundation, ready-made codecs work great with the formats that the client wants to use. All that bothers me is how easy it is to get codecs for it ... Does Windows Media Player Windows 8+ support both DirectShow and MF rendering paths and change them on the fly or does Microsoft completely abandon DirectShow?

I also cannot understand how DXVA should work. Given that I mainly want to visualize video inside the game, it would be wise to do all the rendering actually on the GPU and support the application logic on the processor, but I can’t find good examples of how to use it using the Media Foundation. This is mainly due to the fact that I do not know the difference between DXVA2 and DXVA-HD. The Windows 7 SDK comes with samples for all three implementations ...

If someone can answer my questions in bold, correct my misinformation and call me in the correct example (s) (DirectX + Media Foundation + DXVA-whatever), I would be very grateful! It can be as simple as “combine these 3 examples of the Windows 7 SDK to get what you want).

The purpose of all the assignment is to get the application in top-notch form again, and I want to be absolutely sure that I will choose the methods that will continue over the next decade, like DirectShow: P

Answer 1

If I am correct, are the codecs called MFT? Are there really sites like free-codecs.com?

Codecs, as a rule, are software elements that are either unpacked from a compressed format to uncompressed (decoders), or vice versa (encoders). None of the APIs work exactly with codecs and instead define a common interface for an input and output component, for example, a filter in DirectShow, DirectX Media Object (DMO) or its evolution - Media Foundnation Transform (MFT).

MFT is the basic component of the Media Foundation: encoders, decoders, effects, scaling devices, video capture devices - all of them have MFT inside, and a higher level API is built on their basis.

Due to the limited interest in the Media Foundation, third-party MFTs are rare. Although you can find almost everything in the DirectShow filter form factor, this does not apply to Media Foundation and MFT.

But what is a DXVA-HD? Is it newer than DXVA2, or is it an equivalent name?

DXVA-HD is part of the DXVA 2.0 API family and covers hardware-accelerated video processing. Available as a public low-level interface, it is also integrated into the standard DirectShow and Media Foundation components.

Does the default media player in Windows 8+ use the DirectShow and MF rendering paths and replace them on the fly, or is Microsoft completely abandoning DirectShow?

Yes, WMP 12 still has both. Windows 10 Movies & TV Player is for MF only.

Since I mainly want to render the video in the game, it only makes sense that all the rendering really happens on the GPU, and save the application logic on the CPU, but I can’t find good examples of how to use it using the Media Fund.

Both DirectShow and Media Foundation offer standard decoders with DXVA support (primarily for H.264, but not only). I would say that the DirectShow implementation is rather inflexible and is designed to be used with EVR. Media Foundation is more flexible and can be used outside the MF pipeline as an MFT. Both are done to hide all complexity and use DXVA decoding automatically behind the scenes.

... and I want to be absolutely sure that I will choose the methods that will work over the next decade, as DirectShow already has

DirectShow is still here as a legacy API, with a much more generous offer in examples, code, public information, documentation, third-party extensibility. Media Foundation is a deliberate replacement that attracts very limited interest in it. In fact, it does not offer much compared to DirectShow, Microsoft has not developed it to cover the DirectShow feature set - which is often expected from a newer API, does not offer useful tools for developers. It seemed like MF, as a newbie, was supposed to sell it on his own, but in the end it didn't work:

... The miracle of DirectShow is that although it was written in 1995, this original code still supports the many wild and diverse multimedia formats that have been developed since then. The last base class update that I know of was released in 2009 in the Windows 7 SDK, but even it has minor changes compared to the initial release.

[...]

But remember, even 7 years ago DirectShow was already 15 years old. Base classes were released in source form from the start. The programmers, who care about them, have given them more than two decades of attention. The problems were fixed a long time ago.

[...]

Now, having said all this, it’s true that people in Redmond consider DirectShow a dead technology. They moved to the Media Foundation for reasons that completely avoid me. I still have a problem that Media Foundation solves better than DirectShow. His internal limitations mean that there are problems that he simply cannot solve. I will never understand why they decided to start all over again, instead of correcting what they did not like about DShow. The basic concepts are absolutely identical, although the vocabulary is different. It is just a waste of resources.

July 2019 Update

I would like to relate a comment from Microsoft by James Daily regarding a problem with DirectShow, and it also accurately describes the general status of DirectShow for 2019:

Remember that while DirectShow is technically supported, we are in the process of deprecating this legacy technology. We ask you to switch to MediaFoundation as soon as possible.