Looks like good progress to me!

Quote: 1. There's a flash of magenta / green when the video starts up - that says that the texture is being rendered without being filled, but I black it out as soon as its created, so I'm wondering if the debug runtimes are doing something inside DirectShow too. I only see this on Vista, where there's no DirectDraw (It's emulated through D3D), so it'd make sense if the filter graph is sending a blank frame through. Unfortunately, if it's inside DirectShow, there's not much I can do short of looking at other codecs.

I've seen this mentioned somewhere else in a more user-centric context. Might well be lower level than your app [headshake]

Quote: 3. I only support two video formats; RGB32 and RGB565.

StretchRect() and checking for hardware conversion support?

Quote: and renders the video is a newly created, separate window when rendering with audio (Which is... interesting...).

I saw loads of cases of that Window-in-a-Window and pop-up window type crap with DirectShow/Direct3D integration last time I dealt with it. The ones I could solve involved unholy hacks that I was never happy with [sad]

I'm replying from your previous blog post.

VMR9 stands for Video Mixing Render 9. It's a filter which renders videos via 3D hardware and is available since XP SP2.

http://msdn.microsoft.com/en-us/library/dd407344(VS.85).aspx

It supports three rendering modes: windowed, windowless and renderless.

http://msdn.microsoft.com/en-us/library/dd390956(VS.85).aspx

My DirectShow video renderer uses VMR9 in renderless mode. I decided to implement my code that way because:
- VMR9 is compatibile with XP
- renderless mode should be faster, as my custom allocator/presenter already renders to a proper surface

If you're running vista and you don't build your filter graph by hand, probably your video is rendered by EVR (enanched video render):

http://msdn.microsoft.com/en-us/library/dd375607(VS.85).aspx

Unluckly it is vista only, so I can't use it.

The problems I found with VMR9 are:
- to keep the system flexible enough I partly build my filter graph by hand, letting directshow fill the holes for me. The filter merit system is a nightmare and some codecs generate tons of D3D errors when the video renderer tries to lock a surface.
- on vista (didn't extensively test my code on xp) fullscreen mode is slower than windowed mode (multithreading in an environment where the GPU should be single threaded?)

What's cool is I can render any format WMP plays and I also support positional audio for audio streams. Oh... and I got an mp3 player for free. :)

Sometimes there's an awful combination of cr@p codec/heavy scene putting a lot of pressure on the GPU. In that case the video gets jerky. If the scene framerate is lower than the video framerate that behaviour is predictable, but the video isn't smooth whenever my fps drop below 40-50fps!!!

Last but not least, each codec has a different behaviour when my scene claims too much GPU time. I experienced three different behaviours:
- codec keep trying rendering the next frame despite the running time. This results in video-audio streams desynching (!!!!!!!!!!!).
- codec tries to reach the current position without dropping frames, increasing the video rendering framerate. The effect is "Benny Hill rendering". Synch is nothing but an opinion, as nobody knows if Benny Hill mode will last enough to catch the audio data.
- codec CORRECTLY drop frames, trying to render the frame at the current time. Maybe it's just me but this seems to increases the video jerkiness. My theory is since almost every video compression algorithm is based on differences between frames, to skip seconds requires the video decoder to perform a lot of work to render a single frame.

I'm considering writing an EVR version for vista machines or have a look at Theora.

Any suggestion is welcome!

Quote: Original post by jollyjeffers

Quote: 3. I only support two video formats; RGB32 and RGB565.

StretchRect() and checking for hardware conversion support?

That's a good idea, I'll definitely look into that [smile]

Quote: Original post by jollyjeffers

Quote: and renders the video is a newly created, separate window when rendering with audio (Which is... interesting...).

I saw loads of cases of that Window-in-a-Window and pop-up window type crap with DirectShow/Direct3D integration last time I dealt with it. The ones I could solve involved unholy hacks that I was never happy with [sad]

From what I gather, it's because my renderer isn't suitable, so the filter graph has to create its own to render the file (Like if you drop a .avi file into GraphEdit). It's a DivX file I was testing with, I don't know offhand how that plays into things.

Quote: Original post by undead
I'm replying from your previous blog post.

VMR9 stands for Video Mixing Render 9. It's a filter which renders videos via 3D hardware and is available since XP SP2.

http://msdn.microsoft.com/en-us/library/dd407344(VS.85).aspx

It supports three rendering modes: windowed, windowless and renderless.

http://msdn.microsoft.com/en-us/library/dd390956(VS.85).aspx

My DirectShow video renderer uses VMR9 in renderless mode. I decided to implement my code that way because:
- VMR9 is compatibile with XP
- renderless mode should be faster, as my custom allocator/presenter already renders to a proper surface

If you're running vista and you don't build your filter graph by hand, probably your video is rendered by EVR (enanched video render):

http://msdn.microsoft.com/en-us/library/dd375607(VS.85).aspx

Unluckly it is vista only, so I can't use it.

Ooh, something else for me to look into I think [smile]

Quote: Original post by undead
The problems I found with VMR9 are:
- to keep the system flexible enough I partly build my filter graph by hand, letting directshow fill the holes for me. The filter merit system is a nightmare and some codecs generate tons of D3D errors when the video renderer tries to lock a surface.
- on vista (didn't extensively test my code on xp) fullscreen mode is slower than windowed mode (multithreading in an environment where the GPU should be single threaded?)

I'll see how building my own graph works out for me; surely it's just creating your own video and audio render filters?
I'll have to try out the Vista performance thing and profile it I think.

Quote: Original post by undead
What's cool is I can render any format WMP plays and I also support positional audio for audio streams. Oh... and I got an mp3 player for free. :)

That's something else I was going to investigate actually; How much effort it is to get MP3 playback working with DirectShow...

Quote: Original post by undead
Sometimes there's an awful combination of cr@p codec/heavy scene putting a lot of pressure on the GPU. In that case the video gets jerky. If the scene framerate is lower than the video framerate that behaviour is predictable, but the video isn't smooth whenever my fps drop below 40-50fps!!!

Last but not least, each codec has a different behaviour when my scene claims too much GPU time. I experienced three different behaviours:
- codec keep trying rendering the next frame despite the running time. This results in video-audio streams desynching (!!!!!!!!!!!).
- codec tries to reach the current position without dropping frames, increasing the video rendering framerate. The effect is "Benny Hill rendering". Synch is nothing but an opinion, as nobody knows if Benny Hill mode will last enough to catch the audio data.
- codec CORRECTLY drop frames, trying to render the frame at the current time. Maybe it's just me but this seems to increases the video jerkiness. My theory is since almost every video compression algorithm is based on differences between frames, to skip seconds requires the video decoder to perform a lot of work to render a single frame.

I'm considering writing an EVR version for vista machines or have a look at Theora.

Any suggestion is welcome!

Ah, very cool. I think I'll have a play around with this at lunch time today [smile]

Quote: Original post by Evil Steve
I'll see how building my own graph works out for me; surely it's just creating your own video and audio render filters?

Quote:
That's something else I was going to investigate actually; How much effort it is to get MP3 playback working with DirectShow...

I don't have the code here, but if IIRC you're right: it's all about the graph builder.

You can create a graph builder and attach some filters to it (like VMR9).
When you need to play a media file, the graph builder can fill "holes" in your graph. In my implementation, I want to be sure my graph contains VMR9 and DirectSound filters, while as for the other filters, it's up to the Graph Builder to pick up the appropriate ones. This works and I get extra features for free, like subtitles support if there's a sub file and a sub filter. :)

I say media files and not video files because you can "play" everything. This includes video, video+audio, audio and even pictures (if I send a jpg my media player code will render it to a texture as a one frame movie with no audio data!!! It seems huge images gets cropped, but I never looked at this "issue" in depth, as there's no reason to load textures via directshow! :) ).

What I do is to inspect the media file before creating the graph builder. I analyze its streams and then I compare them with the requested features. If you want to render only video and the input file is an mp3, no graph is built at all as the input file has only an audio stream.

To play an mp3 is quite straightforward, you only add a directsound filter and render the media file. It's also easy to play only video, just add a VMR9 filter.

I wonder how do you handle looping when you play video+audio. In theory you can tell directshow the media content can loop, but IIRC this works only on a few audio formats (wave?). I remember reading articles about messages sent to the application when the stream is complete (EC_COMPLETE message?), but I never liked this "solution". Are you using windows messages?