Capturing, Recording and Streaming Video and Audio from Web-Cams

During long time of development Capture Manager project I have got a significant progress and have got many questions about implementing of it into the other projects. Include open source projects. After some thinking, I have decided to publish source code of Capture Manager SDK under open source license for free using of it by friendly society of software developers. Source code of Capture Manager SDK can be downloaded by link: CaptureManagerSDK-source.zip. 

I would be glad to receive you responses about usefulness of it, you ideas about improving functionality of SDK and recomendation for other developers by the next links:

The most interesting demo programs:

• WPFStreamer (live video and audio streaming on Facebook (SSL encrypted connection: Facebook requires all live video streams to go over RTMPS, or the Real Time-Messaging Protocol (RTMP) on a TLS/SSL connection - this will help protect streaming content via encryption, thereby ensuring a greater level of data integrity) and YouTube sites) - Freeware version.
• WPFRTSPClient - Freeware version.
• WPFRtspServer - Freeware version.

CaptureManager SDK Versions:

• Version 1.20.0

  This release includes updating DotNet upto 4.5 version and implementation of async/await invoking model by the new interfaces of CaptureManager SDK. 

  

  More about async/await you can find in article "Async/Await Explained with Diagrams and Examples" of David Deley.

• Previous versions

CaptureManager help file: 

Download CaptureManagerSDK.chm.zip

NuGet CaptureManager SDK package

Introduction

This article presents my project for capturing of video and audio sources on Windows OS by Microsoft Media Foundation.

I have spent much time on resolving of different tasks with processing of video and audio data, and researched techniques for capturing of video and audio on Windows family OSs. DirectShow was the main technique for capturing live-video on Windows family OSs for long time. However, since Vista OS Microsoft has introduced a new framework for working with video and audio - Microsoft Media Foundation, and since Windows7 OS it has features for working with web-cam via USB Video Class driver and line audio input. I included that technology into many projects and wrote two articles about it: Capturing Live-video from Web-camera on Windows 7 and Windows 8 and Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation. Those articles present simple libraries for capturing web-cams and grabbing of image frames, but Microsoft Media Foundation is much more powerful framework. After some thinking, I have decided to write a new solution for working with live-video and audio which can be more flexible than my previous solutions, and can take much more power (or force :)) of Microsoft Media Foundation. The solution, which can compete with Microsoft solution. The solution, which can be called the best anyone have seen before by any developer. 

Background

I had got an idea to write a new solution for working with web-cams on basement of Microsoft Media Foundation while faced with one unusual task. So, the task was not resolved, but I had wrote some code and had decided to continue development of the solution.

At beginning the solution included only few classes and allowed to execute only few functions, but after adding of some demands for this solution I had decided to write a simple SDK which allows to make capture configuration for the new tasks easy and to inject a new developed code into it by implementation of the Microsoft Media Foundation's and CaptureManager's interfaces. 

As a result, I have got this SDK for capturing, recording and steaming of live-video and audio from web-cams only by Microsoft Media Foundation.

Using the code

CaptureManager SDK Demo programs

OpenGLWebCamViewerViaCOMServer

This demo program presents only simple configuration of the web-cam capturing and viewing via OpenGL rendering.

For using of Capture Manager it needs call the suitable interfaces from CaptureManager COM server. For setting of print output log destination it needs to get ILogPrintOutControl interface:

// get COM log interface
CComPtrCustom<iclassfactory> lCoLogPrintOut;

HRESULT lhresult = CoGetClassObject(CLSID_CoLogPrintOut, CLSCTX_INPROC_SERVER, nullptr, IID_PPV_ARGS(&lCoLogPrintOut));

if (FAILED(lhresult))
    return lhresult;

CComPtrCustom<ilogprintoutcontrol> lLogPrintOutControl;

lCoLogPrintOut->LockServer(true);

lhresult = lCoLogPrintOut->CreateInstance(
    nullptr,
    IID_PPV_ARGS(&lLogPrintOutControl));

if (FAILED(lhresult))
    return lhresult;

// set log file for info
lhresult = lLogPrintOutControl->addPrintOutDestination(
    (DWORD)INFO_LEVEL,
    L"Log.txt");

if (FAILED(lhresult))
    return lhresult;

For getting main methods of the CaptureManager it needs get ICaptureManagerControl interface:

CComPtrCustom<IClassFactory> lCoCaptureManager;

lhresult = CoGetClassObject(CLSID_CoCaptureManager, CLSCTX_INPROC_SERVER, nullptr, IID_PPV_ARGS(&lCoCaptureManager));

if (FAILED(lhresult))
    return lhresult;

lCoCaptureManager->LockServer(true);

// get ICaptureManagerControl interfrace
CComPtrCustom<ICaptureManagerControl> lCaptureManagerControl;

lhresult = lCoCaptureManager->CreateInstance(
    nullptr,
    IID_PPV_ARGS(&lCaptureManagerControl));

if (FAILED(lhresult))
    return lhresult;

The topology of this demo program is presented in the next schema:

This demo program can be gotten by link: Download OpenGLWebCamViewerViaCOMServer.zip

EVRWebCapViewerViaCOMServer

This demo program presents the way of using of Enhanced Video Renderer for rendering of live video via CaptureManager COM server. 

This code allows easy integrate this CaptureManager SDK into already created Windows application - it needs only handler on element of GUI which is used as web-cam display. This code controls resizing of GUI element and modify renderer for changing of video size and proportion.

This demo program can be gotten by link: Download EVRWebCapViewerViaCOMServer.zip

CaptureManagerToCSharpProxy

This demo program presents the way of linking CaptureManager.dll with C# project. In this project CaptureManagerSDK is wrapped by C# class CaptureManager. This class hides direct working with COM interface and marshaling of image data from unmanaged code. For flexibility of solutions the class upload CaptureManager.dll directly into the process and get COM objects WITHOUT of calling of COM's infrastructure - it allows to use CaptureManager COM server WITHOUT any registration in system.

This demo program can be gotten by link: Download CaptureManagerToCSharpProxy.zip

WPFSourceInfoViewer

This demo program presents the way of collecting information from sources and presenting it in the readable form. Information from the sources is presented in form of XML document. There are some reasons for using of XML format for transfering infofmation from COM server:

1. simple structure - Microsoft Media Foundation uses huge amount of GUID constant and types which need to know for understanding of the info, but in this solution all of them presented into the friendly form;

2. easy transfare from COM server;

3. easy parsing - XPath expression allows to get almost any needed information and present it, it is more easy than working with static defined classes and lists of info;

4. In the most cases, user needs to read which features are supported by source and select one of it;

5. XML document is easy integrated source into code of different presentation models (for instance - MVVM).  

This demo program can be gotten by link: 

Download WPFSourceInfoViewer.zip 

Download WPFSourceInfoViewerAsync.zip

WPFWebViewerCall

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by calling of video frames from WPF thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code launchs grabbing of video frames via ISampleGrabber interface and works in sync mode.

This demo program can be gotten by link: 

Download WPFWebViewerCall.zip

Download WPFWebViewerCallAsync.zip

WPFWebViewerCallback

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by calling of video frames from CaptureManager thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code registers update method into the class with ISampleGrabberCallback interface and inject this class into COM Server. While CaptureManager gets new video frame it calls WPF update method and post message to update of frame. In this case, the thread of WPF is not overloaded by grabbing task.

This demo program can be gotten by link: 

Download WPFWebViewerCallback.zip

Download WPFWebViewerCallbackAsync.zip

WPFWebViewerEVR

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by using of Enhanced Video Renderer from CaptureManager thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code gets EVR node from CaptureManager by setting of HWND of the integrated WindowForms panel. In this case, all working with video frames is executed behind of WPF thread.

This demo program can be gotten by link: 

Download WPFWebViewerEVR.zip

Download WPFWebViewerEVRAsync.zip

WPFRecorder

This demo program presents the way of working with CaptureManager SDK for capturing, encodring and recording / broadcasting video - audio from web cameras, microphones, desktop screens and speakers via COM Sever on C# - WPF. 

Code of this demo program presents the correct algorithm of working with CaptureManager SDK via COM Server interfaces. The final point of data stream can be file sink or byte stream sink. The last one is more interesting opportunity then simple saving into the file. In this demo program byte stream sink is implemented in form of TCP Server with Loopback address and http port: 8080. The simular implementation can be find in demo proram on C++, but this one has one significant difference - in this demo program the TCP Server is implemented on C#-dotNet framework. CaptureManager SDK is written on C++ and uses Windows Media Foundation on C, however, there is a good news - CaptureManager SDK needs implementation Windows Media Foundation interface IMFByteStream for streaming of bytes. Implementation C interface IMFByteStream on C# does not need any specific Windows Media Foundation function - it needs defining interfaces: IMFByteStream, IMFAsyncCallback, IMFAsyncResult; and enum constants: 

• MFAsyncCallbackQueue, 
• MFASync, 
• MFByteStreamSeekingFlags, 
• MFByteStreamSeekOrigin, 
• MFByteStreamCapabilities, 
• UnmanagedNameAttribute.  

Implementation of those interfaces can be found in file NetworkStreamSink.cs, but I would like to point your attention on the next moments:

1. in IMFByteStream interface there are four implemented methods: GetCapabilities, Write, BeginWrite, EndWrite. In the first one the code sets type of implementation of  IMFByteStream interface - writteable, but not seekable. Method  Write is used for sync writing data into the stream, while methods BeginWrite, EndWrite are used for async writing. However, there are some important moments: method  Write is called once ant the start - it writes the head of medta streams: type of encoder, amount of streams, names of streams and other metadata. Async writing needs to execute the methods in the next order:  BeginWrite, argument IMFAsyncCallback pCallback.Invoke, EndWrite, but Calling of methods BeginWrite, EndWrite can be locked by the same mutex. It means that argument IMFAsyncCallback pCallback.Invoke must be executed in the separated thread - for example by ThreadPool.QueueUserWorkItem.  

2. In the implementation of TCP Server I have used async calling BeginAcceptTcpClient, and writing head data at the start of each connection - it allows to connect any mount of client to the media stream server. 

public void Start()
{
try
{
tcpListener = new TcpListener(Configuration.IPAddress, Configuration.Port);
tcpListener.Start();
tcpListener.BeginAcceptTcpClient(
new AsyncCallback(callBack),
tcpListener);
}
catch (Exception e)
{
}
}

/// <summary>
/// Stops the WebServer thread
/// </summary>

public void Stop()
{
try
{
tcpListener.Stop();
foreach (var item in mClientBag)
{
item.Value.Client.Close();
item.Value.Client.Dispose();
item.Value.Close();
}
tcpListener.Server.Dispose();
}
catch (Exception e)
{
}
}

private void callBack(IAsyncResult aIAsyncResult)
{
TcpListener ltcpListener = (TcpListener)aIAsyncResult.AsyncState;
if (ltcpListener == null)
return;
TcpClient lclient = null;
try
{
lclient = ltcpListener.EndAcceptTcpClient(aIAsyncResult);
}
catch (Exception exc)
{
return;
}
if (lclient != null && lclient.Client.Connected)
{
StreamReader streamReader = new StreamReader(lclient.GetStream());

// Read full request with client header
StringBuilder receivedData = new StringBuilder();
while (streamReader.Peek() > -1)
receivedData.Append(streamReader.ReadLine());
string request = GetRequest(receivedData.ToString());
if (!SuportedMethod(request))
{
SendError(StatusCode.BadRequest, "Only GET is supported.", lclient);
lclient.Client.Close();
lclient.Close();
}
else
{
Socket socket = lclient.Client;
if (socket.Connected)
{
SendHeader(StatusCode.OK, lclient);
lock (this)
{
if (mHeaderMemory != null)
{
int sentBytes = socket.Send(mHeaderMemory);
}
mClientBag[lclient] = lclient;
}
}
}
}
ltcpListener.BeginAcceptTcpClient(
new AsyncCallback(callBack),
ltcpListener);
}

3. The head includes MIME type of byte stream, which allows to use in the future release the same solution for any type of media container - ASF, MP4, MKV.

private void SendHeader(string mimeType, long totalBytes, StatusCode statusCode, TcpClient aTcpClient)
{
StringBuilder header = new StringBuilder();
header.Append(string.Format("HTTP/1.1 {0}\r\n", GetStatusCode(statusCode)));
header.Append(string.Format("Content-Type: {0}\r\n", mimeType));
header.Append(string.Format("Accept-Ranges: bytes\r\n"));
header.Append(string.Format("Server: {0}\r\n", Configuration.ServerName));
header.Append(string.Format("Connection: close\r\n"));
header.Append(string.Format("Content-Length: {0}\r\n", totalBytes));
header.Append("\r\n");

SendToClient(header.ToString(), aTcpClient);
}

This demo program can be gotten by link: 

Download WPFRecording.zip

Download WPFRecordingAsync.zip

QtMinGWDemo

This demo program presents the way of working with CaptureManager SDK for capturing, encoding and recording / broadcasting live video and audio from web cameras, microphones, desktop screens and speakers via COM Sever on Qt framework. On Windows OS there is a version of Qt for Visual Studio Compiler, but this demo presents using of CaptureManager SDK with Qt version for MinGW Compiler. Of course, this demo can be recompiled for Visual Studio Compiler, but version for MinGW Compiler shows flexibility of this SDK and compaliblness with many other compilers. This demo include code for capturing live video from web cameras and viewing by calling of samples, by callback of view update code from Capture Manager SDK inner thread or by direct drawing image via HWND of widget.

Another example in this demo presents way for connection of sources, encoders and sinks into the one pipe line of processing. This demo presents way for recording video and audio into the file and real code for writing of network stream broadcasting into the Internet code by Qt framework QTcpServer  and QTcpSocket classes.

This demo program can be gotten by link: Download QtMinGWDemo. 

CaptureManagerSDKPythonDemo

This demo program presents the way of working with CaptureManager SDK for capturing of live video from web cameras in Python application on Windows OS. CaptureManager is built with using of Microsoft Media Foundation on Desktop Windows OSs and COM technology. As any COM server CaptureManager can be intergated into the project via direct calling interfaces or type library. However, dynamic type programming languages have some problems with correct using type libraries. For resolving such problems I have included IDispatch and wrote implementation for many classes, but some projects need working with pointers on massive of memory and it could be a problem with dynamic types. For simplifing solution I have implemented IDispatch interface only for limited functionality. This demo program presents functionality for selections of sources, encoders and rendering video via HWND and saving into the file,      

This demo program can be gotten by link: Download CaptureManagerSDKPythonDemo

WPFWebCamShot

This demo program presents the way of working with CaptureManager SDK for capturing single frame from video stream. CaptureManager SDK includes new mode for SampleGrabberCallSinkFactory - PULL.

This mode is differ from SYNC and ASYNC modes. SYNC and ASYNC modes work in continues mode - they send request for next sample after receiving of the current one in automatic mode without blocking. It leads queue of requests and samples like on image:

This type is useful for presenting of video, but the task of capturing the single frame has faced with the next dificulty - 99 percents of frames are captured, decoded and delivered to the sink, but customer's code does not take it from sink. This task can be in reality - for example, taking single frame each one second while camera produces 30 frames per second - it this case CPU wastes time and power on 29 frames which will throw out into the garbge. For such task new mode - PULL can be more useful. The sink sends request only while customer's code calls for the new single frame:

It allows utilize CPU power more effective only for one task. The new mode can be useful also for many projects of image recognition - such projects usually process frames with frame rate low than it can be produced by cameras, but video pipeline will spend much CPU power on frames, which will not be processed.  PULL mode allows release some CPU power from video pipeline for other parts of program.    

This demo program can be gotten by link: 

Download WPFWebCamShot.zip

Download WPFWebCamShotAsync.zip

WPFWebCamSerialShots

This demo program presents the way of working with CaptureManager SDK for capturing the last serial frames from video stream. CaptureManager SDK 1.2.0 includes new type of stream control node - sample accumulator node: 

It includes two sample accumulator nodes on 5 and 10 samples which collect/accumulate 5 or 10 LAST SAMPLES. Idea is that in the real world taking of photo does not executed in the moment of viewing of event. Human reaction on event, process of pressing on take photo button, sending event from hardware layer into layer of customer's code and requesting of the single frame take some time - from 500 ms til 1 second. It leads to loose of important moment which was the reason of taking of photo. Sample accumulator nodes can compensate such time delay by accumulating the last samples in video stream. These samples can be gotten by PULL mode of SampleGrabberCallSinkFactory. It sends the request:

and receives the last samples:

It can be useful for resolving of some problems, but accumulation of samples needs memory buffer which can be expanded fast - for example for keeping of 10 last samples of RGB32 image of 1080p format it needs about 60 MByte - one second accumulation in such video can need about 200 MByte or more. Of course, it is possible create sample accumulator with different schema - for example one by third - it means keeping only each third sample from video stream.

This demo program can be gotten by link: 

Download WPFWebCamSerialShots.zip

Download WPFWebCamSerialShotsAsync.zip

CaptureManagerToJavaProxy

This demo program presents the way of linking CaptureManager.dll with Java VM. This project presents Java wrapper of COM CaptureManager via calling JNI functions. It includes CaptureManagerToJavaProxy framework for reflection Java code on C code of CaptureManagerNativeProxy.dll. CaptureManagerToJavaProxy framework includes code for calling native code from JAR - it allows build runnable JAR file with packed CaptureManagerNativeProxy.dll and CaptureManager.dll. This project uses Microsoft Media Foundation - it limitates Java projects ONLY Windows OSs since from Windows 7.  

This proxy programs can be gotten by links: 

Download CaptureManagerToJavaProxy.zip,

Download CaptureManagerJNIProxy.zip

CaptureManagerSDKJavaxDemo

This demo program presents the way of working with web camera on WindowsOSs via Java VM. This demo program allows present live video from web camera via Javax.swing GUI framework. It allows get HWND descriptor of GUI component and use EVR for rendering without marshaling images via Java VM JNI - it allows safe much CPU power:

In addition, this demo program includes functionality for recording video from web camera into the ASF video file:

You can test this demo program by compiling runnable JAR file or downloading CaptureManagerSDKJavaxDemo.jar.zip which contains CaptureManagerSDKJavaxDemo.jar. It could be important point your attention on architecte of VM - x86 and x86-64. In the most situations Java compiled code can be executed on both architectes. However, CaptureManagerToJavaProxy calls native code via JNI - it leads the next problems: Native DLL with JNI for x86 cannot be uploaded into x86-64 Java VM, and Native DLL with JNI for x86-64 cannot be uploaded into x86 Java VM. These problems are resolved by finding of Java VM architecte at runtime and uploading CaptureManagerNativeProxy.dll of the suitable architecte into the Java VM. It means that JAR includes TWO CaptureManagerNativeProxy.dll and TWO CaptureManager.dll.

I think that you will find its very suitable - of course, it is workable ONLY on WindowsOSs since Windows 7.

This demo program can be gotten by links: 

Download CaptureManagerSDKJavaxDemo.zip

Download CaptureManagerSDKJavaxDemo.jar.zip

WindowsFormsDemo:

This demo program presents the way of working with web camera on WindowsOSs on WindowsForms GUI.

This demo program allows present live video from web camera via WindowsForms GUI framework. It allows get HWND descriptor of GUI component and use EVR for rendering without marshaling data via C# - it allows safe much CPU power: 

This demo shows how to select different types of sources. At the present time, CaptureManager supports three source types: Webcam, Screen, DirectShow Crossbar. These sources are different, and it is possible have connected some of these source type devices at the same time. It leads to need a solution for flexible selecting the suitable group of source type. The next code of generating XPath query implements such functionality in this demo:

string lXPath = "//*[";

if(toolStripMenuItem1.Checked)
{

    lXPath += "Source.Attributes/Attribute[@Name='MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY']/SingleValue[@Value='CLSID_WebcamInterfaceDeviceCategory']";
}
                        
if (toolStripMenuItem2.Checked)
{
    if (toolStripMenuItem1.Checked)
        lXPath += "or ";

    lXPath += "Source.Attributes/Attribute[@Name='MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_HW_SOURCE']/SingleValue[@Value='Software device']";
}

if (dSCrossbarToolStripMenuItem.Checked)
{
    if (toolStripMenuItem1.Checked || toolStripMenuItem2.Checked)
        lXPath += "or ";

        lXPath += "Source.Attributes/Attribute[@Name='MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY']/SingleValue[@Value='CLSID_VideoInputDeviceCategory']";
}                      
            
lXPath += "]";

In addition, this demo includes code for recording video and audio from media sources:

This demo program can be gotten by link: 

Download WindowsFormsDemo.zip

Download WindowsFormsDemoAsync.zip

WaterMarkInjectorDemo:

This demo program presents the way of working of custom Microsoft Media Foundation Transform with CaptureManager. This demo program allows 'inject' image into the live video via special custom Microsoft Media Foundation Transform which works like 'effect filter'. 

This demo program can be gotten by link: Download WaterMarkInjector.zip

TextInjectorDemo:

This demo program presents the way of working of custom Microsoft Media Foundation Transform with CaptureManager. This demo program allows 'inject' text string into the live video via special custom Microsoft Media Foundation Transform which works like 'effect filter' and change its content. 

The text string content can be changed from application side in continues mode with displaying result immediately. In this demo program it is done by the next code:

wchar_t ltext[MAXBYTE];

/* program main loop */
while (!bQuit)
{
	/* check for messages */
	if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
	{
		/* handle or dispatch messages */
		if (msg.message == WM_QUIT)
		{
			bQuit = TRUE;
		}
		else
		{
			TranslateMessage(&msg);
			DispatchMessage(&msg);
		}
	}

	_itow_s(++lFrameCount, ltext, 10);

	lITextWriter->writeText(ltext);

	Sleep(200);
}

This demo program can be gotten by link: Download TextInjector.zip

WPFImageViewer:

This demo program presents the way of working with ICaptureProcessor, IInitilaizeCaptureSource,  ICurrentMediaType, ISourceRequestResult interfaces of CaptureManagerToCSharpProxy. This demo program implements ICaptureProcessor interface in ImageCaptureProcessor class - this class allows 'inject' customized data into the capture session. ICaptureProcessor interface has methods: 

void initilaize(IInitilaizeCaptureSource IInitilaizeCaptureSource),

void pause(),

void setCurrentMediaType(ICurrentMediaType aICurrentMediaType),

void shutdown(),

void sourceRequest(ISourceRequestResult aISourceRequestResult),

void start(long aStartPositionInHundredNanosecondUnits, ref Guid aGUIDTimeFormat),

void stop().

Method void initilaize(IInitilaizeCaptureSource IInitilaizeCaptureSource) must set in IInitilaizeCaptureSource argument XML text string with the description of capture source - this data is got from image info. Method void setCurrentMediaType(ICurrentMediaType aICurrentMediaType)  allows select correct stream index and correct media type. Method void sourceRequest(ISourceRequestResult aISourceRequestResult) is executed by CaptureManage SDK for getting of raw data in format which is defined in XML text string with the description. The format of such XML document has the next form:

<!--?xml version='1.0' encoding='UTF-8'?-->
<presentationdescriptor streamcount="1">
	<presentationdescriptor.attributes title="Attributes of Presentation"> 
		<attribute description="Contains the unique symbolic link for a video capture driver." guid="{58F0AAD8-22BF-4F8A-BB3D-D2C4978C6E2F}" name="MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_SYMBOLIC_LINK" title="The symbolic link for a video capture driver.">
			<singlevalue value="ImageCaptureProcessor">
		</singlevalue></attribute>
		<attribute description="The display name is a human-readable string, suitable for display in a user interface." guid="{60D0E559-52F8-4FA2-BBCE-ACDB34A8EC01}" name="MF_DEVSOURCE_ATTRIBUTE_FRIENDLY_NAME" title="The display name for a device."> 
			<singlevalue value="Image Capture Processor">
		</singlevalue></attribute>
	</presentationdescriptor.attributes> 
	<streamdescriptor index="0" majortype="MFMediaType_Video" majortypeguid="{73646976-0000-0010-8000-00AA00389B71}"> 
		<mediatypes typecount="1"> 
			<mediatype index="0">
				<mediatypeitem description="Width and height of a video frame, in pixels." guid="{1652C33D-D6B2-4012-B834-72030849A37D}" name="MF_MT_FRAME_SIZE" title="Width and height of the video frame."> 
					<value.valueparts> 
						<valuepart title="Width" value="Temp_Width"> 
						<valuepart title="Height" value="Temp_Height"> 
					</valuepart></valuepart></value.valueparts>
				</mediatypeitem>
				<mediatypeitem description="Approximate data rate of the video stream, in bits per second, for a video media type." guid="{20332624-FB0D-4D9E-BD0D-CBF6786C102E}" name="MF_MT_AVG_BITRATE" title="Approximate data rate of the video stream."> 
					<singlevalue value="33570816">
				</singlevalue></mediatypeitem>
				<mediatypeitem description="The major type defines the overall category of the media data." guid="{48EBA18E-F8C9-4687-BF11-0A74C9F96A8F}" name="MF_MT_MAJOR_TYPE" title="Major type GUID for a media type."> 
					<singlevalue guid="{73646976-0000-0010-8000-00AA00389B71}" value="MFMediaType_Video"> 
				</singlevalue></mediatypeitem>
				<mediatypeitem description="Default surface stride, for an uncompressed video media type. Stride is the number of bytes needed to go from one row of pixels to the next." guid="{644B4E48-1E02-4516-B0EB-C01CA9D49AC6}" name="MF_MT_DEFAULT_STRIDE" title="Default surface stride."> 
					<singlevalue value="Temp_Stride"> 
				</singlevalue></mediatypeitem>
				<mediatypeitem description="Specifies for a media type whether the samples have a fixed size." guid="{B8EBEFAF-B718-4E04-B0A9-116775E3321B}" name="MF_MT_FIXED_SIZE_SAMPLES" title="The fixed size of samples in stream."> 
					<singlevalue value="True"> 
				</singlevalue></mediatypeitem>
				<mediatypeitem description="Frame rate of a video media type, in frames per second." guid="{C459A2E8-3D2C-4E44-B132-FEE5156C7BB0}" name="MF_MT_FRAME_RATE" title="Frame rate."> 
					<ratiovalue value="10.0"> 
						<value.valueparts> 
							<valuepart title="Numerator" value="10">  
							<valuepart title="Denominator" value="1">  
						</valuepart></valuepart></value.valueparts> 
					</ratiovalue> 
				</mediatypeitem>
				<mediatypeitem description="Pixel aspect ratio for a video media type." guid="{C6376A1E-8D0A-4027-BE45-6D9A0AD39BB6}" name="MF_MT_PIXEL_ASPECT_RATIO" title="Pixel aspect ratio."> 
					<ratiovalue value="1"> 
						<value.valueparts> 
							<valuepart title="Numerator" value="1">  
							<valuepart title="Denominator" value="1">  
						</valuepart></valuepart></value.valueparts> 
					</ratiovalue> 
				</mediatypeitem>
				<mediatypeitem description="Specifies for a media type whether each sample is independent of the other samples in the stream." guid="{C9173739-5E56-461C-B713-46FB995CB95F}" name="MF_MT_ALL_SAMPLES_INDEPENDENT" title="Independent of samples."> 
					<singlevalue value="True">  
				</singlevalue></mediatypeitem>
				<mediatypeitem description="Specifies the size of each sample, in bytes, in a media type." guid="{DAD3AB78-1990-408B-BCE2-EBA673DACC10}" name="MF_MT_SAMPLE_SIZE" title="The fixed size of each sample in stream."> 
					<singlevalue value="Temp_SampleSize">  
				</singlevalue></mediatypeitem>
				<mediatypeitem description="Describes how the frames in a video media type are interlaced." guid="{E2724BB8-E676-4806-B4B2-A8D6EFB44CCD}" name="MF_MT_INTERLACE_MODE" title="Describes how the frames are interlaced."> 
					<singlevalue value="MFVideoInterlace_Progressive">  
				</singlevalue></mediatypeitem>
				<mediatypeitem description="The subtype GUID defines a specific media format type within a major type." guid="{F7E34C9A-42E8-4714-B74B-CB29D72C35E5}" name="MF_MT_SUBTYPE" title="Subtype GUID for a media type."> 
					<singlevalue guid="{Temp_SubTypeGUID}">
				</singlevalue></mediatypeitem>
			</mediatype>
		</mediatypes>
    </streamdescriptor>
</presentationdescriptor>

Temp_SubTypeGUID defines image format, Temp_SampleSize is byte size of image, Temp_Stride is stride in byte or 0 for compressed formats, Temp_Width and Temp_Height are pixel width and height of image.

This demo program can be gotten by link: 

Download WPFImageViewer.zip

Download WPFImageViewerAsync.zip

WPFIPCameraMJPEGViewer:

This demo program presents the way of working with ICaptureProcessor, IInitilaizeCaptureSource,  ICurrentMediaType, ISourceRequestResult interfaces of CaptureManagerToCSharpProxy. This demo program implements ICaptureProcessor interface in IPCameraMJPEGCaptureProcessor class - this class allows 'inject' MJPEG frames from IP camera. This class connects to the selected web camera by HTTP web socket and redirect the received data into the capture session. ICaptureProcessor inteface has methods: 

void initilaize(IInitilaizeCaptureSource IInitilaizeCaptureSource),

void pause(),

void setCurrentMediaType(ICurrentMediaType aICurrentMediaType),

void shutdown(),

void sourceRequest(ISourceRequestResult aISourceRequestResult),

void start(long aStartPositionInHundredNanosecondUnits, ref Guid aGUIDTimeFormat),

void stop().

 Method void initilaize(IInitilaizeCaptureSource IInitilaizeCaptureSource) must set in IInitilaizeCaptureSource argument XML text string with the description of capture source - this data is got from image. Method void setCurrentMediaType(ICurrentMediaType aICurrentMediaType)  allows select correct stream index and correct media type. Method void sourceRequest(ISourceRequestResult aISourceRequestResult) is executed by CaptureManage SDK for getting of raw data in format which is defined in XML text string with the description. 

This demo program can be gotten by link: 

Download WPFIPCameraMJPEGViewer.zip

Download WPFIPCameraMJPEGViewerAsync.zip

WPFViewerEVRDisplay:

This demo program presents the way of working with  IEVRMultiSinkFactory   and IEVRStreamControl interfaces of CaptureManagerToCSharpProxy. This demo program shows way for creating of the EVR sink from Direct3DSurface9 inteface. WPF allows present DirectX texture as regular image via System.Windows.Interop.D3DImage class. However, the demand for compatibility on wide range Windows OSs allows use only old DirectX technique - DirectX9. This code creates render target texture of Direct3DSurface9 inteface and CaptureManager uses this texture a destination surface. This solution allows easy integrate rendering result with the WPF GUI components. 

This demo program shows how to use scaling and setSrcPosition method of IEVRStreamControl interface for effective implementation of zooming and defining Region Of Interest (ROI) and controlling of them.

setSrcPosition method has argumets Left, Right, Top, Bottom which define corners of ROI Rectangle in relative value from 0.0f to 1.0f. This demo uses Screen Capture source for correct working without accessable USB cameras. 

This demo program can be gotten by link: 

Download WPFViewerEVRDisplay.zip

Download WPFViewerEVRDisplayAsync.zip

WPFMultiSourceViewer:

This demo program presents the way of working with  IEVRMultiSinkFactory and IEVRStreamControl interfaces of CaptureManagerToCSharpProxy. This demo program shows way for creating of two EVR sinks and controlling of them.

This demo shows how connect them with different capture sessions and control of destination: size, position, scale, flush. This demo uses Screen Capture sources for correct working without accessable USB cameras. 

This demo program can be gotten by link: 

Download WPFMultiSourceViewer.zip

Download WPFMultiSourceViewerAsync.zip

WPFIPCameraMJPEGMultiSourceViewer:

This demo program presents the way of working with ICaptureProcessor, IInitilaizeCaptureSource, ICurrentMediaType, ISourceRequestResult,  IEVRMultiSinkFactory and IEVRStreamControl interfaces of CaptureManagerToCSharpProxy. This demo program shows way for creating of two EVR sinks, controlling of them and connecting of them with two instances of IPCameraMJPEGCaptureProcessor class.

This demo program can be gotten by link: 

Download WPFIPCameraMJPEGMultiSourceViewer.zip

Download WPFIPCameraMJPEGMultiSourceViewerAsync.zip

EVRVieweingAndRecording:

This demo program presents the way of working with CaptureManager SDK for capturing of live video and audio from sources, encoding them and recording them into the file with preview of live video. Connecting of components is presented on the next schema:

 Demo program has consol inteface:

This demo program can be gotten by link: Download EVRVieweingAndRecording.zip

WPFVideoAndAudioRecorder:

This demo program presents the way of working with CaptureManager SDK for capturing of live video and audio from sources, encoding them and recording them into the file with preview of live video. Connecting of components is presented on the next schema:

Demo program has WPF GUI and allows to control more correctly:

This demo program can be gotten by link: 

Download WPFVideoAndAudioRecorder.zip

Download WPFVideoAndAudioRecorderAsync.zip

NativeMediaFoundationPlayer:

This demo program presents the way of working with CaptureManager SDK for rendering video from media file by connecting it to the code of MediaFoundation player. The original video renderer of MediaFoundation has a limited functionality and can be used for simple programms,  but for more complex solutions it needs develop a renderer with the needed functionality. CaptureManager version 1.7.0 Freeware has supporting video rendering with the next functionality:

1. MultiSink video rendering - it means that it is possible render more than one video stream on the one video context. Each video rendering sink has independent control from others. 

2. It can use one of three video context:

      2.1 Handler on window which is rendering target:

 

      2.2 DirectX9 Texture - render target texture which can be used for texturing of mesh for final rendering:

      2.3 DirectX9 SwapChain - render target texture from swap chain of DirectX9 renderer:

3. it allows control position, size and background color of rendering.

This demo program can be gotten by link: Download NativeMediaFoundationPlayer.zip

WPFMediaFoundationPlayer:

This demo program presents the way of working with CaptureManager SDK for rendering video from media file by connecting it to the code of MediaFoundation player.

This demo program has WPF interface and show how correct create group of MediaFoundation players with CaptureManager MultiSink video renderers. It supports CaptureManager version 1.7.0 Freeware:

This demo program can be gotten by link: 

Download WPFMediaFoundationPlayer.zip

Download WPFMediaFoundationPlayerAsync.zip

UnityWebCamViewer

This demo program presents the way of working with CaptureManager SDK in Unity Game Development SDK on Windows OSs. CaptureManager SDK is included into the Unity Game Development SDK by plugin which allows move pointer of DirectX9 Rendering Texture into the C++ code for setting it like rendering target. C# code of the Unity Game Development SDK gets text string from CaptureManager SDK and displays information about sources.   

This demo program can be gotten by links:

plugin - CaptureManagerWebCamViewerPlugin.zip,

Unity project - UnityWebCamViewer.zip,

WPFMultiSourceRecorder

This demo program presents the way for recording of video and audio data from many sources into the one file. CaptureManager supports flexible management of connection of many sources to many sinks. ASF sink implements Windows OS supporting of ASF Microsoft media format - special format for Microsoft platforms with many features. This demo program allows test one of these features - multi stream recording. It is possible select many sources for recording. More over, these sources can include more than one video source. It allows record syncronized video from many live video sources into the one media file.

Schema of source connection has the next view:

VideoLAN player can open such multi video stream file:

This demo program can be gotten by link: 

Download WPFMultiSourceRecorder.zip

Download WPFMultiSourceRecorderAsync.zip

WPFScreenViewer

This demo program presents the way for modify "Screen Capture" source by adding of "normalize" mode. CaptureManager has functionality for capturing image from the screen of display. CaptureManager supports multi-display outputs and can create live video stream from "Screen Capture" source with two modes: regular and normalize. "Regular" mode is mode by default for "Screen Capture" source - it takes image from displays as-is - whout any post processing:

"Landscape" image is taken in normal view:

"Flipped Landscape" image is taken as image rotated on 180 degrees:

"Portrait" image is taken as image rotated on 90 degrees:

"Flipped Portrait" image is taken as image rotated on 270 degrees:

It is done for keeping of the maximum quality fo video encoding.

"Normalize" mode allows overcome problem with different proportion of screens by executing of image post processing operations. At the present moment it supports one normalize - "Landscape" - " --normalize=Landscape" - it normalizes any image to the one type of orientation - top in upper part of image, bottom in lower part of image. it can be done in code:

            if ((bool)mNormalizeChkBtn.IsChecked)
                lextendSymbolicLink += " --normalize=Landscape";

Enabling this mode leads to the next result:

"Landscape" image is taken in normalize post processing:

"Flipped Landscape" image is taken in normalize post processing:

"Portrait" image is taken in normalize post processing:

"Flipped Portrait" image is taken in normalize post processing:

This demo program can be gotten by link: 

Download WPFScreenViewer.zip

Download WPFScreenViewerAsync.zip

WPFScreenRecorder

This demo program presents the way for setting of the "Screen Capture" source's options. 

 "Cursor" option's extension is expanded by new attribute "Shape" - it has the next values:

• "Rectangle" - shape of filling is rectangle
• "Ellipse" - shape of filling is ellipse

It allows set the rectangle shape of back image:

or ellipse shape of back image:

"Clip" option allows define "Region Of Iterest" on the desktop screen and clip it to the output video stream:

XML of then new option has the next form:

<Option Type='Clip' >
    <Option.Extensions>
        <Extension Left='0' Top='0' Height='100' Width='100' />
    </Option.Extensions>
</Option>

This demo program can be gotten by link: 

Download WPFScreenRecorder.zip

Download WPFScreenRecorderAsync.zip

WPFWindowScreenRecorder

This demo program presents the way for setting of the "Screen Capture" source's options for capturing screen view of the selected application's window. 

Selecting of the source application window is started by click on "Select by click" button - it initiliazes Windows OS API for getting HWND handler of window which is under mouse pointer - the name of window which is under of the mouse pointer is displayed under "Select by click" button:

Catching of the HWND handler of the window is done by pressing of "L" keyboard button. It allows get size of the catching window and select frame rate:

After selection of the window, it is possible apply "Screen Capture" source's options for setting of the cursor back mask and cliping region of interest:

Setting of HWND of the source screen view window is done by adding argument "--HWND" to "Screen Capture" symbolic link:

lextendSymbolicLink += " --HWND=" + m_CurrentHWND.ToInt32().ToString();

This demo program can be gotten by link: 

Download WPFWindowScreenRecorder.zip

Download WPFWindowScreenRecorderAsync.zip

UnityScreenRecorder

This demo program presents the way of working with ICaptureProcessor interface of CaptureManager SDK in Unity Game Development SDK on Windows OSs. CaptureManager SDK is included into the Unity Game Development SDK by plugin which allows capture game engine's rendered screen and record it into the output video file.

  

This demo program supports two mode of working: 

First mode - capture and render image for checking functionality of the correct working of the capture processor:

Second mode - recording with the selection of encoder and output format file:

This demo program can be gotten by links:

plugin - CaptureManagerUnityScreenRecorderPlugin.zip,

Unity project - UnityScreenRecorder.zip,

UnityVideoAndAudioRecorder

This demo program presents the way of viewing and recording video and audio from external sources in Unity Game Development SDK on Windows OSs.

This demo program can be gotten by links:

plugin - CaptureManagerUnityVideoAndAudioRecorder.zip,

Unity project - UnityVideoAndAudioRecorder.zip,

WPFViewerTrigger

This demo program presents the way of using of the Switcher node. This code uses two output nodes for getting images from the source: EVR and Sample Grabber. Sample Grabber takes sample with 1 second timing and checks average brightness in sample - if this value is more or equal threshold, then Switcher node is set into turn on and EVR gets sampes with 30 fps. If this value is less threshold, then Switcher node is set into turn off and EVR gets nothing. This schema has the next view:

The result demo has the next view:

This demo program can be gotten by link: 

Download WPFViewerTrigger.zip

Download WPFViewerTriggerAsync.zip

WPFPauseResumeRecording

This demo program presents the way of using of the Switcher node. Switcher is set in recording stream and allows pause/resume this video stream while EVR preview stream still work in independent continue mode. 

This demo program can be gotten by link: 

Download WPFPauseResumeRecording.zip

Download WPFPauseResumeRecordingAsync.zip

WPFHotRecording

This demo program presents the way of using of the Switcher node for attaching and detaching of the recording stream to the source at runtime WITHOUT stoping of the source. ISwitcherControl interface has commands for flexible changing configuration of the Switcher node.

Such solution can be useful for task of recording media files with the fixed size - while size of the current recording media file reaches the threshold value then the current file is detached and the new one is attached to the recording stream.

In this demo program the Switcher nodes (for video and audio streams) are created WITHOUT recording media file - it is possible. The new recording media file is attached by clicking on "Start recording" button. The old one (if it is exist) is detached.

This demo program can be gotten by link: 

Download WPFHotRecording.zip

Download WPFHotRecordingAsync.zip

WPFAreaScreenRecorder

This demo program presents the way of using of the Screen Capture Source. Some time ago I needed screen recorder application and tried develop simple application which can cover almost 99% functionality of the other similar applications, but use only CaptureManager SDK. It has simple interface:

It has clipping by "elastic band rectacle":

It generates filename by current date:

It has configuration for showing and hiding cursor icon and back image mask:

It has configuration for video encoder with setting compression quality:

It has configuration selection output file format and selection directory for storing the recorded video files:

Current configurations can be saved and reseted to the default values.

This demo program can be gotten by link: 

Download WPFAreaScreenRecorder.zip

Download WPFAreaScreenRecorderAsync.zip

InterProcessRenderer

These demo programs present the way of using of the DirectX9 texture shared handler for sharing of the rendering context between two processes. They include WPFInterProcessClient and InterProcessRenderer projects. 

Direct3D9Ex allows create DirectX9 texture with the shared handler:

private Direct3DSurface9 GetSharedSurface(Direct3DDevice9Ex device)
{
    return device.CreateRenderTarget(
           m_width,
           m_height,
           Format, // D3DFMT_A8R8G8B8
           0,
           0,
           0  // UNLOCKABLE
           );

}

Each Direct3DSurface9 has Handle property which can be used for the rendering target in another process. 

Such solution allows to separate user interface process and capture-rendering process:

mOffScreenCaptureProcess = new Process();

mOffScreenCaptureProcess.StartInfo.FileName = "InterProcessRenderer.exe";

mOffScreenCaptureProcess.EnableRaisingEvents = true;

mOffScreenCaptureProcess.Exited += mOffScreenCaptureProcess_Exited;

mPipeProcessor = new PipeProcessor(
    "Server",
    "Client");

mPipeProcessor.MessageDelegateEvent += lPipeProcessor_MessageDelegateEvent;

mOffScreenCaptureProcess.StartInfo.Arguments =
    "SymbolicLink=" + aSymbolicLink + " " +
    "StreamIndex=" + aStreamIndex + " " +
    "MediaTypeIndex=" + aMediaTypeIndex + " " +
    "WindowHandler=" + mVideoPanel.Handle.ToInt64();

mOffScreenCaptureProcess.StartInfo.UseShellExecute = false;
            
try
{
    mIsStarted = mOffScreenCaptureProcess.Start();

    HeartBeatTimer.Start();
}
catch (Exception)
{
    mIsStarted = false;
}

This demo programs can be gotten by link: 

Download InterProcessRenderer.zip

Download InterProcessRendererAsync.zip

WPFDeviceInfoViewer

This demo program presents the way for grouping of the video and audio sources by unique device id. Each device in Windows OS has unique device id for identity each one. This demo program displays this id and groups devices according of this id.

This solution can be usefull for task with many video and audio USB sources from one vender - this code allows group video and audio sources according to belong of the real USB device.

This demo program can be gotten by link: 

Download WPFDeviceInfoViewer.zip

Download WPFDeviceInfoViewerAsync.zip

WPFWebViewerEVRHandler

This demo program presents the way of using of the DirectX9 texture shared handler for rendering. The current generation of Windows OS has functionlaity for sharing of the rendering texture between different versions of DirectX. 

This demo program can be gotten by link: 

Download WPFWebViewerEVRHandler.zip

Download WPFWebViewerEVRHandlerAsync.zip

DShowPlayer

This demo program present the way for using of CaptureManager SDK with DirectShow player as a Video Renderer in C++ project of Window application. This demo program uses DirectShow for playing video in C++ code, and CaptureManager's EVRMultiSink output node:

    HRESULT hr = S_OK;

    IBaseFilter *pEVR = NULL;

    //hr = AddFilterByCLSID(pGraph, CLSID_EnhancedVideoRenderer, &pEVR, L"EVR");

    unsigned long lPtrOutputNodeAmount = 0;

    CaptureManagerLoader::getInstance().getMaxVideoRenderStreamCount(&lPtrOutputNodeAmount);

    std::vector<CComPtrCustom<IUnknown>> lOutputNodes;

    CaptureManagerLoader::getInstance().createRendererOutputNodes(
        hwnd,
        nullptr,
        1,
        lOutputNodes);
    
    if (!lOutputNodes.empty())
    {
        lOutputNodes[0]->QueryInterface(IID_PPV_ARGS(&pEVR));
    }
        
    CHECK_HR(hr = pGraph->AddFilter(pEVR, L"EVR"));

This demo program can be gotten by link: Download DShowPlayer.zip

WPFDirectShowPlayer

This demo program present the way for using of CaptureManager SDK with DirectShow player as a Video Renderer in WPF application. This demo program uses DirectShowLib C# NuGet for playing video in C# code, and CaptureManager's EVRMultiSink output node:

            List<object> lOutputNodesList = new List<object>();

            lCaptureManagerEVRMultiSinkFactory.createOutputNodes(
                IntPtr.Zero,
                mEVRDisplay.Surface.texture,
                1,// lMaxVideoRenderStreamCount,
                out lOutputNodesList);

            if (lOutputNodesList.Count == 0)
                return;

            IBaseFilter lVideoMixingRenderer9 = (IBaseFilter)lOutputNodesList[0];

            var h = m_pGraph.AddFilter(lVideoMixingRenderer9, "lVideoMixingRenderer9");

Version for DirectShow has only software implementation of renderer.

This demo program can be gotten by link: 

Download WPFDirectShowPlayer.zip

Download WPFDirectShowPlayerAsync.zip

WPFRtspServer

This demo program present the way for using of CaptureManager SDK in RTSP streaming service. This demo program is based on GitHub project SharpRTSP - C# version of a RTSP handling library for development of the RTSP communication server and client. 

CaptureManager SDK has functionality for getting raw data from the video and audio sources - demo program WPFWebViewerCallback presents the way for getting of RGB32 images for displaying. However, this SDK allows get any raw data from the SDK pipeline - ever of the encoded video and audio data. This demo program defines output node for video stream:

                ISampleGrabberCallback lH264SampleGrabberCallback;

                aISampleGrabberCallbackSinkFactory.createOutputNode(
                    MFMediaType_Video,
                    MFVideoFormat_H264,
                    out lH264SampleGrabberCallback);

                object lOutputNode = lEVROutputNode;

                if (lH264SampleGrabberCallback != null)
                {
                    uint ltimestamp = 0;

                    lH264SampleGrabberCallback.mUpdateEvent += delegate
                        (byte[] aData, uint aLength)
                    {
                        if (s != null)
                        {
                            ThreadPool.QueueUserWorkItem((object state) =>
                            {
                                s.sendData(aIndexCount, ltimestamp, aData);

                                ltimestamp += 33;
                            });
                        }
                    };

and for audio stream:

                ISampleGrabberCallback lAACSampleGrabberCallback;

                aISampleGrabberCallbackSinkFactory.createOutputNode(
                    MFMediaType_Audio,
                    MFAudioFormat_AAC,
                    out lAACSampleGrabberCallback);

                if (lAACSampleGrabberCallback != null)
                {
                    lAACSampleGrabberCallback.mUpdateFullEvent += delegate
                        (uint aSampleFlags, long aSampleTime, long aSampleDuration, byte[] aData, uint aLength)
                    {
                        if (s != null)
                        {
                            ThreadPool.QueueUserWorkItem((object state) =>
                            {
                                s.sendData(aIndexCount, (uint)aSampleTime / 100000, aData);
                            });
                        }
                    };

Combination of this SDK with code of the SharpRTSP allows create workable version of the RTSP Video and Audio Streaming Server: 

For supporting of the wide spread decoder CaptureManager allows encode video into the H264 and H265 (HEVC) video formats:

This demo program can be gotten by link: 

Download WPFRtspServer.zip

Download WPFRtspServerAsync.zip

WPFRTSPClient

This demo program present the way for using of CaptureManager SDK in RTSP streaming service. This demo program is based on GitHub project SharpRTSP - C# version of a RTSP handling library for development of the RTSP communication server and client. 

CaptureManager SDK has functionality for capturing raw dat from an external source. This source can be encoded data - in this example, the source is a test server with H264 video encoded stream:

This demo program can be gotten by link: 

Download WPFRTSPClient.zip

Download WPFRTSPClientAsync.zip

WPFStreamer

This demo program presents the way for using of CaptureManager SDK in RTMP (Real-Time Messaging Protocol (RTMP) was initially a proprietary protocol developed by Macromedia for streaming audio, video and data over the Internet, between a Flash player and a server. Macromedia is now owned by Adobe, which has released an incomplete version of the specification of the protocol for public use) streaming services (Facebook (RTMPS), YouTube and others). This demo program is based on librtmp - C version of a RTMP handling library for development of the RTMP streaming client. 

This demo program can be gotten by link: 

Download WPFStreamer.zip

Download WPFStreamerAsync.zip

WPFMixer

This demo program presents the way for using mixering functionality of CaptureManager SDK. New SDK interfaces:

• IMixerNodeFactory,
• IVideoMixerControl,
• IAudioMixerControl

allow to combine many video streams into the one video stream and many audio streams into the one audio stream. Interface IMixerNodeFactory creates list of topology input nodes with the one output node. Interface IVideoMixerControl allows control video mixing of each source: setPosition - set position of the input video stream on the back reference video stream, setSrcPosition - set region in input video stream, setZOrder - set Z order of mixer, setOpacity - set opacity of input video stream, flush - clear video input stream buffer.

public interface IVideoMixerControl
{
    bool setPosition(object aPtrVideoMixerNode,
        /* [in] */ float aLeft,
        /* [in] */ float aRight,
        /* [in] */ float aTop,
        /* [in] */ float aBottom);
        
    bool setSrcPosition(object aPtrVideoMixerNode,
        /* [in] */ float aLeft,
        /* [in] */ float aRight,
        /* [in] */ float aTop,
        /* [in] */ float aBottom);
        
    bool setZOrder(object aPtrVideoMixerNode,
        /* [in] */ UInt32 aZOrder);

    bool setOpacity(object aPtrVideoMixerNode,
        /* [in] */ float aOpacity);
        
    bool flush(object aPtrVideoMixerNode);
}

Interface IAudioMixerControl allows control mixing audio stream with the reference back audio stream: setRelativeVolume - set volume of the audio stream to relative for reference back audio stream = (input audio stream * aRelativeVolume) + (reference back audio stream * (1.0 - aRelativeVolume)).

public interface IAudioMixerControl
{
    bool setRelativeVolume(object aPtrAudioMixerNode, float aRelativeVolume);
}

This demo program can be gotten by link: 

Download WPFMixer.zip

Download WPFMixerAsync.zip

WPFGIFtoVideo

This demo program presents the way for recording animated gif images into the video file by CaptureManager SDK. For injection decoded GIF frames demo program uses implemented interface ICaptureProcessor. 

This demo program can be gotten by link: 

Download WPFGIFtoVideo.zip

Download WPFGIFtoVideoAsync.zip

AudioCaptureProcessor

This demo program presents the way for recording output from system audio mixe into the audio file by CaptureManager SDK. Audio data is type of data which needs low latency for correct recording - CaptureManager SDK has optimized media processing pipeline and can capture audio data with high quality. 

This demo program can be gotten by link: Download AudioCaptureProcessor.zip

WPFStreamingAudioRenderer

This demo program presents the way for playing audio stream from webcamera into the system audio renderer by CaptureManager SDK. 

This demo program can be gotten by link: 

Download WPFStreamingAudioRenderer.zip

Download WPFStreamingAudioRendererAsync.zip

CaptureManager SDK Versions:

Version 1.0.0

Capture Manager SDK v1.0.0 – open release. I include interfaces with the stable definition:

via class CoLogPrintOut – interface ILogPrintOut

via class CoCaptureManager – interface ICaptureManagerControl:

method createControl – create the main control objects with interfaces:

• ISourceControl,
• ISinkColtrol,
• IEncoderControl,
• ISessionControl,
• IStreamControl;

method createMisc – create a miscellaneous object with interfaces:

• IMediaTypeParser,
• IStrideForBitmap,
• IVersionControl;

via object with interface ISourceControl:

• IWebCamControl;

via object with interface ISinkControl:

• IFileSinkFactory,
• ISampleGrabberCallSinkFactory,
• IEVRSinkFactory,
• ISampleGrabberCallbackSinkFactory,
• IByteStreamSinkFactory;

via object with interface ISession:

• ISessionCallback;

via object with interface ISampleGrabberCallSinkFactory:

• ISampleGrabberCall;

via object with interface ISampleGrabberCallbackSinkFactory:

• ISampleGrabberCallback;

These interfaces do the next work:

• ILogPrintOut – it manages writing of log information;
• ICaptureManagerControl – it manages creating of all capturing controls and miscellaneous objects;
• ISourceControl – it manages and creates sources of video and audio signals;
• ISinkColtrol – it manages and creates sinks of video and audio streams;
• IEncoderControl – it manages and creates video and audio encoders;
• ISessionControl – it creates object for management of recording session;
• IStreamControl – it creates object for controlling of media streams;
• IMediaTypeParser – it creates text representation of media type in XML format;
• IStrideForBitmap – it computes stride of memory for the specific bitmap format;
• IVersionControl – it manages information about current version of CaptureManager;
• IWebCamControl – it manages options of web camera;
• IFileSinkFactory – it creates media sink nodes which are linked with media file;
• ISampleGrabberCallSinkFactory – it creates media sink for grabbing of one sample by direct calling object with interface ISessionCallback;
• ISampleGrabberCall – it manages grabbing of one sample;
• ISampleGrabberCallbackSinkFactory – it creates media sink for grabbing of one sample by calling object with interface ISampleGrabberCallback from CaptureManager inner thread;
• ISampleGrabberCallback – it manages grabbing of one sample from CaptureManager inner thread;
• IEVRSinkFactory – it creates media sink for rendering of video stream;
• IByteStreamSinkFactory – it creates media sink nodes which are linked with customised byte stream object with interface IMFByteStream;
• ISession – it manages the recording session;
• ISessionCallback – it manages the result state of current recording session from CaptureManager inner thread;

The definitions of these interfaces are presented in SDK files and in CaptureManagerToCSharpProxy project. For effective marshalling information via border of unmanaged-managed code CaptureManager uses eight simple XML documents with the next structures:

<!--?xml version="1.0" ?-->
<!--XML Document of sources-->

These documents contain information about sources, encoders, sinks, stream controls. Most types and GUID is taken from Microsoft Media Foundation MSDN: Microsoft Media Foundation, but there are some rules for definition of nodes in these XML documents: SingleValue - node for presenting only one value - name, integer, type; RatioValue - node for presenting of value in float point format with Numerator and Denominator; Value.ValueParts - node for storing collection of ValuePart nodes with the same type. Examples of parsing of these XML documents can be found in code of WPF examples.

Capture Manager SDK can be downloaded by link: Download CaptureManagerSDK.zip

Version 1.1.0 with HEVC(H265) encoder

Capture Manager SDK v1.1.0 – open release. This release includes the next changings:

• Add supporting of automating registration CaptureManager as COM server with Type Library
• Add supporting of dynamic language Python 2.7
• Add supporting of HEVC(H265) encoder in Windows 10: Download HEVCEncoder_Windows10

Capture Manager SDK v1.1.0 can be downloaded by link: Download CaptureManagerSDK-110.zip

Version 1.2.0 Beta

Capture Manager SDK v1.2.0 – beta release. This release includes the next changings:

• Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
• Lazy binding of Microsoft Media Foundation functions.
• Replace web camera properties functionality from DirectShow on DeviceIoControl. 
• Add resizing in EVR. 

Lazy binding of functions is implemented by loading of Microsoft Media Foundation libraries at runtime. Functions which cannot be found in Microsoft Media Foundation libraries are replaced by stub functions. It allows prevent crushing of applications by using of unsupported functions of Microsoft Media Foundation libraries.

Using of DeviceIoControl for changing of web camera properties allows resolve problem with delay of web camera driver initialization. It allows expand workable set of web camera properties by : "Amount of digital zoom", "Upper limit for the amount of digital zoom", "Power line frequency".

Resizing in EVR resolves problem with unchanging image size of rendered video. The new implementation controls current image size of renderer GUI and changes image size and position for keeping proportion of video.

Capture Manager SDK v1.2.0 – beta release can be downloaded by link: Download CaptureManagerSDK-1.2.0_beta.zip

Version 1.2.0

Capture Manager SDK v1.2.0 – release. This release includes the next changings:

• Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
• Lazy binding of Microsoft Media Foundation functions.
• Replace web camera properties functionality from DirectShow on DeviceIoControl. 
• Add resizing in EVR.
• Add PULL mode in SampleGrabberCallSinkFactory - the new mode which allows to take a single sample.
• Add sample accumulator nodes for storing of 5 or 10 last samples in media stream.

Lazy binding of functions is implemented by loading of Microsoft Media Foundation libraries at runtime. Functions which cannot be found in Microsoft Media Foundation libraries are replaced by stub functions. It allows prevent crushing of applications by using of unsupported functions of Microsoft Media Foundation libraries.

Using of DeviceIoControl for changing of web camera properties allows resolve problem with delay of web camera driver initialization. It allows expand workable set of web camera properties by : "Amount of digital zoom", "Upper limit for the amount of digital zoom", "Power line frequency".

Resizing in EVR resolves problem with unchanging image size of rendered video. The new implementation controls current image size of renderer GUI and changes image size and position for keeping proportion of video.

Capture Manager SDK v1.2.0 – release can be downloaded by link: Download CaptureManagerSDK-1.2.0-Freeware.zip

Version 1.3.0 Beta with DirectShow Crossbar capture

Capture Manager SDK v1.3.0 – beta. This release includes the next changings:

• Add supporting of video capturing via DirectShow Crossbar technique for the next inputs:                                  Composite,
       SVideo,
       USB,
       1394 (FireWire).

On MSDN - Audio/Video Capture in Media Foundation you can find that "Video capture devices are supported through the UVC class driver and must be compatible with UVC 1.1". It means that Microsoft Media Foundation can capture video only from devices with supporting of UVC driver - USB Video Class driver. It is usually USB web cameras - other types of video capture devices are not supported. From my experience I can say that the target platform of Microsoft Media Foundation is WindowsStore applications. WindowsStore applications are originally targeted on portable devices like "Pad" - "Surface" - such devices DO NOT have external ports for connecting of capturing cards. They can work only with embedded web cameras which are connected via inner USB ports. I think that it is a correct idea, but I have got many questions about using of CaptureManager with capture cards which support DirectShow Crossbar and answered that it is impossible. However, it is not full answer - the full answer is that it is possible see capture card which supports DirectShow Crossbar via Microsoft Media Foundation functionality, but captured video is not accessable. After some research I found the way to get access for such capture cards. Info for working with capture card can be selected from XML string of sources via checking of attributes nodes of Source with name 'MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY': "Source.Attributes/Attribute[@Name='MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY']

/SingleValue[@Value='CLSID_VideoInputDeviceCategory']" - Microsoft Media Foundation define CLSID_WebcamInterfaceDeviceCategory - {E5323777-F976-4F5B-9B55-B94699C46E44} for device with UVC supporting and CLSID_VideoInputDeviceCategory - {860BB310-5D01-11D0-BD3B-00A0C911CE86} for DirectShow capture devices. It allows easy select only DirectShow video capture devices. Input connection of DirectShow according to physical type can be selected by chosing of the suitable media type - each media type node has child MediaTypeItem node with name DirectShowPhysicalType - {09FCAE6B-8F00-4C0A-BBCA-C6820B576C99}. It allows to group the meda types according physical type of input connection.

<MediaTypeItem Name="DirectShowPhysicalType" GUID="{09FCAE6B-8F00-4C0A-BBCA-C6820B576C99}" Title="Physical type of input pin of Crossbar." Description="Physical type of input pin of Crossbar.">
  <SingleValue Value="PhysConn_Video_Composite" /> 
  </MediaTypeItem>

Capture Manager SDK v1.3.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware-beta.zip

Version 1.3.0 with DirectShow Crossbar capture

Capture Manager SDK v1.3.0. This release includes the next changings:

• Add supporting of video capturing via DirectShow Crossbar technique for the next inputs:                                  Composite,
       SVideo,
       USB,
       1394 (FireWire);
• COM ThreadingModel has been changed from Apartment on Both.

Capture Manager SDK v1.3.0 can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware.zip

Version 1.4.0 Beta

Capture Manager SDK v1.4.0 – beta. This release includes the next changings:

• Add supporting for changing of cursor presenting in Screen Capture source: make cursor invisible and add drawing additional image for current cursor's image: 

This functionality is gotten by adding " --options=" into the symbolic link. Options is set by XML format:

<!--?xml version='1.0' encoding='UTF-8'?-->
<Options>
    <Option Type="Cursor" Visiblity="True">
        <Option.Extensions>
            <Extension Fill="0x7055ff55" Height="100" Type="BackImage" Width="100"/>
        </Option.Extensions>
    </Option>
</Options>

Selection of cursor option is done by "Type='Cursor'" attribute. Attribute "Visiblity='True'" makes current type - Cursor visible, and attribute "Visiblity='False'" sets current type - Cursor invisible - it allows capture screen without drawing of cursor. XML node "Option.Extensions" allows add extensions.  At the present time, Screen capture source supports the next extexnsion: 

"<Extension Type='BackImage' Height='100' Width='100' Fill='0x7055ff55' />"

Attribute "Type='BackImage'" defines drawing before cursor's image 'marker' image:

Size of 'BackImage' is defined by attributes "Height='100' Width='100'", color is defined by attribute "Fill='0x7055ff55'" with ARGB format. In code, it can be implemented by this way:

string lextendSymbolicLink = lSymbolicLink + " --options=" +
    "<?xml version='1.0' encoding='UTF-8'?>" +
        "<Options>" +
            "<Option Type='Cursor' Visiblity='True'>" +
                "<Option.Extensions>" +
                    "<Extension Fill='0x7055ff55' Height='100' Type='BackImage' Width='100'>" +
                    "</Extension> +
                    </Option.Extensions>" +
            "</Option>" +
        "</Options>";

Capture Manager SDK v1.4.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.4.0-Freeware-beta.zip

Version 1.4.0

Capture Manager SDK v1.4.0. This release includes the next changings:

• Add supporting for changing of cursor presenting in Screen Capture source: make cursor invisible and add drawing additional image for current cursor's image.
• Add supporting for using of custom Microsoft Media Foundation Transform for implementation custom effects (for example - WaterMarkInjectorDemo, TextInjectorDemo).

Capture Manager SDK v1.4.0 can be downloaded by link: Download CaptureManagerSDK-1.4.0-Freeware.zip

Version 1.5.0

Capture Manager SDK v1.5.0. This release includes the next changings:

• Add supporting of multidisplay outputs on: 

 

• Add supporting of DirectShow software filters.

Capture Manager SDK v1.5.0 can be downloaded by link: Download CaptureManagerSDK-1.5.0-Freeware.zip

Version 1.6.0

Capture Manager SDK v1.6.0. This release includes the next changings:

• Add IEVRMultiSinkFactory, IEVRStreamControl interfaces: IEVRMultiSinkFactory allows create two separated Topology Nodes for the one rendering contex - HWND of Window, Direct3DSurface9 or IDXGISwapChain. IEVRStreamControl allows control of rendering result - position rendering, Z-order rendering, flushing of stream buffer, filters and features.
• IInitilaizeCaptureSource, ICurrentMediaType, ISourceRequestResult, ICaptureProcessor interfaces:  IInitilaizeCaptureSource receives configuration of Capture Processsor. ICurrentMediaType contains info of the selected Media Type. ISourceRequestResult receives raw data in format which is defined in IInitilaizeCaptureSource. ICaptureProcessor controls the capturing process.     

CaptureManager has implementation of multisink rendering via sharing context. This solution is different from common implementation of Microsoft Media Foundation. Original solution uses 'Session' pattern - working with sources, decoders, encoders and sinks in the one session. As a result, it leads to syncronize of many stream pipelines from the one session and they can be controlled by one command `Start`, `Stop`, `Pause`.  It works fine in case of playing of one video file. Microsoft Media Foundation has function for creating one source from many others - MFCreateAggregateSource - it allows to implement effect `Picture in Picture` for rendering two and more video files. However, it has limitating - it is workble only in context of the one session - it means that all sources of the session started, stopped and paused at the same time.

It can be OK for playing of video from files, but what to do in the situation of sources with different latency - for two sources - one from local webcam and another from IP camera by HTTP. Another problem - if we want stop only one source the code stops ALL sources in the session. The solution of these problems can be using of DIFFERENT INDEPENDENT sessions for each source, but common implementation of Microsoft Media Foundation DOES NOT allows share video rendering sink via many sessions. CaptureManager resolves such problems by adding one level abstraction - while common implementation of Microsoft Media Foundation creates ONE Video Rendering Media Sink from the one video context, CaptureManager creates Many Video Rendering Media Sinks from Controller which shares the one video context.

Additional level allows manage rendering more flexible. Demo program for testing of such functionality can be found by link WPFMultiSourceViewer. CaptureManager Freeware supports maximum 2 rendering sinks from the one rendering context. MultiRendering presented in `Sink Factories` by new `Sink Factory` node:

<sinkfactory guid="{10E52132-A73F-4A9E-A91B-FE18C91D6837}" name="EVRMultiSinkFactory" title="Enhanced Video Renderer multi sink factory">
  <value.valueparts>
   <valuepart description="Default EVR implementation" guid="{E926E7A7-7DD0-4B15-88D7-413704AF865F}" maxportcount="2" mime="" title="Container format" value="Default"> 
  </valuepart>
  </value.valueparts>
</sinkfactory>

Rendering can be controlled by IEVRStreamControl interface:

• setPosition method allows positioning of rendering on the shared rendering context in relative values from 0.0f to 1.0f;
• getPosition method allows get positioning of rendering on the shared rendering context in relative values from 0.0f to 1.0f;
• setZOrder method allows set rendering order of stream from 0 to max rendering sinks;
• getZOrder method allows get endering order of stream from 0 to max rendering sinks;
• flush method allows clear rendering buffer of the rendering sink;
• setSrcPosition method allows set rectangle in the input stream for selection of Region Of Interest;
• getSrcPosition method allows get rectangle in the input stream for selection of Region Of Interest;
• getCollectionOfFilters method allows get XML string with list of filters for current rendering sink:

<!--?xml version="1.0" ?--> 
<!-- XML Document of render stream filters --> 
  <filters> 
    <filter currentvalue="" max="" min="" step="" title=""> 
  </filter></filters> 

• setFilterParametr method allows set filter for current rendering sink;
• getCollectionOfOutputFeatures method allows get XML string with list of output features for rendering context:

<!--?xml version="1.0" ?-->
<!-- XML Document of render stream output features -->
<features>
<feature title="Background Color">
<color>
<channel currentvalue="0" index="1" max="255" min="0" step="1" title="Red">
<channel currentvalue="0" index="2" max="255" min="0" step="1" title="Green">
<channel currentvalue="0" index="3" max="255" min="0" step="1" title="Blue">
</channel></channel></channel></color>
</feature>
</features>

• setOutputFeatureParametr method allows set output feature for rendering context;

CaptureManager has functionality for capturing data from custom Capture Processor. It includes ICaptureProcessor interface for implemeting custom CaptureProcessor class. The needed configuration is set in argument of initilaize method with IInitilaizeCaptureSource interface. The selected configuration is set in argument of setCurrentMediaType method with ICurrentMediaType interface. The needed data is requested in sourceRequest method by argument with ISourceRequestResult inteface. It allows to develop solutions for capturing of raw data WPFImageViewer or WPFIPCameraMJPEGViewer.

CaptureManager SDK v1.6.0 can be downloaded by link: Download CaptureManagerSDK-1.6.0-Freeware.zip

Version 1.7.0

Capture Manager SDK v1.7.0. This release includes the next changings:

• Add CaptureManager Video Renderer Factory which is compatable with MediaFoundation player. It supports  IEVRMultiSinkFactory, IEVRStreamControl interfaces via additional sink factory:  

<SinkFactory Name="CaptureManagerVRMultiSinkFactory" GUID="{A2224D8D-C3C1-4593-8AC9-C0FCF318FF05}" Title="CaptureManager Video Renderer multi sink factory">
<Value.ValueParts>
<ValuePart Title="Container format" Value="Default" MIME="" Description="Default EVR implementation" MaxPortCount="2" GUID="{E926E7A7-7DD0-4B15-88D7-413704AF865F}" /> 
 </Value.ValueParts>
</SinkFactory>

Add the new demo programs:

CaptureManager Video Renderer Factory can be used for implementing of video sink for rendering video from media file in many solutions

CaptureManager SDK v1.7.0 can be downloaded by link: Download CaptureManagerSDK-1.7.0-Freeware.zip

Version 1.8.0

Capture Manager SDK v1.8.0. This release includes the next changings:

• Add "normalize" mode for the "Screen Capture" source.
• Add IRenderingControl interface for controlling of rendering on target process.  
• Add "Shape" attribute for "Cursor" option of "Screen Capture" source.
• Add "Clip" type option of "Screen Capture" source.

Add the new demo programs:

The "Screen Capture" source supports capture of screen as-is - it means that the rotated image is got as it is created by computer - rotated. However, in some situation it can need get image in some specific orientation of image, for example get "Portrait" orentated screen in "Landscape" form. For this purpose, the "Screen Capture" source supports mode "normalize" for post processing of the current image to the needed orientation:  " --normalize=Landscape".

The "Screen Capture" source supports additional shape of the back image - "Ellipse", and the new option "Clip":

The new IRenderingControl interface is created from ICaptureManagerControl::createMisc method and expands controlling of the rendering process.  IRenderingControl interface has the next signature:

    MIDL_INTERFACE("1CBCAF1C-1809-41DE-A728-23DBF86A6170")
    IRenderingControl : public IDispatch
    {
    public:
        virtual /* [id][helpstring] */ HRESULT STDMETHODCALLTYPE enableInnerRendering( 
            /* [in] */ IUnknown *aPtrEVROutputNode,
            /* [in] */ BOOL aIsInnerRendering) = 0;
        
        virtual /* [id][helpstring] */ HRESULT STDMETHODCALLTYPE renderToTarget( 
            /* [in] */ IUnknown *aPtrEVROutputNode,
            /* [in] */ IUnknown *aPtrRenderTarget,
            /* [in] */ BOOL aCopyMode) = 0;
        
    };

Method enableInnerRendering allows switch rendering model in the CaptureManager. Method renderToTarget allows execute rendering to the target texture from user rendering thread. 

The reason to include a new interface for controlling of rendering is cased by thread-safe of DirectX. In the previous versions of CaptureManager rendering executed in the context of the inner thread of CaptureManager. In the most situation such way for rendering does not lead to problem - in the demo programs the DirectX9 device is created with supporting of multithreading and DirectX11device is thread-safe by default. However, I faced with problem of rendering while have started modify UnityWebCamViewer demo project for including of DirectX11 rendering. After some search in code I found that ID3D11Device::CreateDeferredContext return error DXGI_ERROR_INVALID_CALL - according of https://msdn.microsoft.com/en-us/library/windows/desktop/ff476505(v=vs.85).aspx - "if the device was created with the D3D11_CREATE_DEVICE_SINGLETHREADED value,CreateDeferredContext returns DXGI_ERROR_INVALID_CALL." It means that Unity engine does not support updating texture from none rendering thread.   IRenderingControl interface allows switch from multithreading rendering to single thread rendering mode. By default, CaptureManager has multithreading rendering which can be switched to single thread rendering by method enableInnerRendering with FALSE state of aIsInnerRendering. Method renderToTarget allows render to target in context of the single rendering thead - for target can be used rendering texture with interface IDirect3DTexture9 for DirectX9 or with interface ID3D11Texture2D for DirectX11. Argument aCopyMode must be set to FALSE for DirectX9. For DirectX11, argument aCopyMode can be TRUE or FALSE - it depends from type of rendering texture. DirectX11 allows create TYPELESS texture formats for flexibility converting of types. However, DirectX11 Video processor supports only UNORM types of rendering texture. Unity engine creates rendering texture with TYPELESS texture formats - it leads creating of errors. Argument aCopyMode with TRUE allows create additional texture with UNORM type and use this texture for target of rendering and copy from this texture to the original rendering texture with TYPELESS texture format. This operation is executed in context of GPU very fast. 

There is CaptureManager help file: Download CaptureManagerSDK.chm.zip

Capture Manager SDK v1.8.0 can be downloaded by link: Download CaptureManagerSDK-1.8.0-Freeware.zip

Version 1.9.0

Capture Manager SDK v1.9.0. This release includes the next changings:

• Add "HWND" mode for the "Screen Capture" source.

Add the new demo programs:

Capture Manager SDK v1.9.0 can be downloaded by link: CaptureManagerSDK-1.9.0-Freeware.zip

Version 1.10.0

Capture Manager SDK v1.10.0. This release includes the next changings:

• Add ISwitcherNodeFactory interface for pause/resume of the media stream.
• Add ISwitcherControl interface for controling (pause, resume, detach, attach) media streams.

Add the new demo programs:

Capture Manager SDK v1.10.0 can be downloaded by link: CaptureManagerSDK-1.10.0-Freeware.zip

Version 1.11.0

Capture Manager SDK v1.11.0. This release includes the next changings:

• Add DirectX11 Texture Capture.
• Change Screen Capture output format from RGB32 to NV12.

Capture Manager SDK v1.11.0 can be downloaded by link: CaptureManagerSDK-1.11.0-Freeware.zip

Version 1.12.0

Capture Manager SDK v1.12.0. This release includes the next changings:

• Add CM_DEVICE_LINK attribute with unique device id;
• Support DirectX9Ex texture shared handler.

Add the new demo programs:

Capture Manager SDK v1.12.0 can be downloaded by link: CaptureManagerSDK-1.12.0-Freeware.zip

Version 1.13.0

Capture Manager SDK v1.13.0. This release includes the next changings:

• Add supporting of DirectShow Video Renderer.

Add the new demo programs:

Capture Manager SDK v1.13.0 can be downloaded by link: CaptureManagerSDK-1.13.0-Freeware.zip

Version 1.14.0

Capture Manager SDK v1.14.0. This release includes the next changings:

• Add H.264/MPEG-4 AVC video stream encoder (from 256 kbit/c to 64 kbit/c);
• Add H.265/MPEG-4 HEVC video stream encoder (64 kbit/c);
• Add AAC audio stream encoder (96 kbit/c);
• Add video stream decoder.

Add the new demo programs:

• Add WPFStreamer demo program;
• Add WPFRTSPClient demo program;
• Add WPFRtspServer demo program.

Capture Manager SDK v1.14.0 can be downloaded by link: CaptureManagerSDK-1.14.0-Freeware.zip

Version 1.15.0

Capture Manager SDK v1.15.0. This release includes the next changings:

• Add fast SSE2/SSSE3/SSE4.2 copy.

Capture Manager SDK v1.15.0 can be downloaded by link: CaptureManagerSDK-1.15.0-Freeware.zip

Version 1.16.0

Capture Manager SDK v1.16.0. This release includes the next changings:

• Add interfaces for mixering video and audio streams:
  • IMixerNodeFactory,
  • IVideoMixerControl,
  • IAudioMixerControl.

Add the new demo programs:

• Add WPFMixer demo program;
• Add WPFGIFtoVideo demo program;
• Add AudioCaptureProcessor demo program;

Capture Manager SDK v1.16.0 can be downloaded by link: CaptureManagerSDK-1.16.0-Freeware.zip

Version 1.17.0

Capture Manager SDK v1.17.0. This release includes the next changings:

• Expand H264 and H265 video encoder bitrate upto 2^25 (33,554,432)

Capture Manager SDK v1.17.0 can be downloaded by link: CaptureManagerSDK-1.17.0-Freeware.zip

Version 1.18.0

Capture Manager SDK v1.18.0. This release includes the next changings:

• Add Audio Renderer.

Add the new demo programs:

• Add WPFStreamingAudioRenderer demo program.

Capture Manager SDK v1.18.0 can be downloaded by link: CaptureManagerSDK-1.18.0-Freeware.zip

Version 1.19.0

Capture Manager SDK v1.19.0. This release includes the next changings:

• Add supporting of H264 webcam.

Capture Manager SDK v1.19.0 can be downloaded by link: CaptureManagerSDK-1.19.0-Freeware.zip

Version 1.19.1

Capture Manager SDK v1.19.1. This release includes the next changings:

• Fix bug with pull mode.
• Fix bug with open camera already in use.

Capture Manager SDK v1.19.1 can be downloaded by link: CaptureManagerSDK-1.19.1-Freeware.zip

Version 1.20.0

Capture Manager SDK v1.20.0. This release includes the next changings:

• Add async/await invoking model.

Capture Manager SDK v1.20.0 can be downloaded by link: CaptureManagerSDK-1.20.0-Freeware.zip 

Points of Interest

Previously I wrote that the was one unusual task which was the reason to start of development of CaptureManager. The task included recording live-video of industrial process from two sources and recording of data from two sensors. I have started to write a new solution for Microsoft Media Foundation and thought to use Microsoft ASF format due to existence of MFMediaType_Binary Major Type. However, after some time I found that implementation of Microsoft ASF format for Microsoft Media Foundation for recording supports only MFMediaType_Video,  MFMediaType_Audio,  and MFMediaType_Script. It was the main reason for stopping to resolve that task.           

Updates:

The first update on 21/08/2015:

improved quality of the AudioLoopback capture;

resolved problem with syncronization of the Screen Capture Source - GDIScreenCapture and AudioLoopback;

added new Media Types for Screen Capture Source - GDIScreenCapture with the next frame rates: 20 fps, 25 fps and 30 fps;

added into Screen Capture Source - GDIScreenCapture supporting for capturing of cursor and drawing it into the captured video.

The second update on 31/08/2015:

fixed loosing of samples in the AudioLoopback capture;

added new Media Types for Screen Capture Source - GDIScreenCapture with the next frame rates: 1 fps, 5 fps and 10 fps;

added supporting of WM Speech Encoder DMO;

added supporting of WMVideo9 Screen Encoder MFT - FOURCC "MSS2";

added version of CaptureManager SDK for x64 Windows OS.

The third update on 14/09/2015:

added new Screen Capture Source - DirectX9ScreenCapture with the next frame rates: 1 fps, 5 fps, 10 fps, 15fps, 20 fps, 25 fps, 30 fps;

added library CaptureManagerProxy.dll for calling CaptureManagerSDK descriptors in "C" style;

added supporting of 3rdParty MediaSink solutions - RecordInto3rdPartyMediaSink;

added demo program for capturing of screen in DirectX9 video games - ScreenCaptureWithDirect3D9APIHooks;

The fourth update on 30/09/2015:

added supporting of Enhanced Video Renderer - the demo program: EnhancedVideoRendererWebCapViewer

The fifth update on 13/11/2015:

added limited supporting of COM Server;

added CaptureManager.tlb;

added demo programs for presenting of working with CaptureManagerSDK via COM on C#-WPF: 

CaptureManagerToCSharpProxy,

WPFSourceInfoViewer,

WPFWebViewerCall,

WPFWebViewerCallback,

WPFWebViewerEVR

The sixth update on 07/03/2016:

first stable release 1.0.0;

implemented supporting of all SDK functionality in COM Server;

wrote new COM Server interface;

stopped development of C interface;

replace GDIScreenCapture and DirectX9ScreenCapture on the Screen Capture;

included full support of play, pause, stop and close functinality;

updated code of demo programs for presenting of working with CaptureManagerSDK via COM on C#-WPF: 

CaptureManagerToCSharpProxy,

WPFSourceInfoViewer,

WPFWebViewerCall,

WPFWebViewerCallback,

WPFWebViewerEVR; 

added new demo program for demonstration of functinality of recording and network streaming media content:

WPFRecorder;

The seventh update on 21/03/2016:

Add demo program for working with Capture Manager SDK in Qt framework programs on MinGW compiler: QtMinGWDemo

The eighth update on 12/04/2016

Add demo program for working with CaptureManager SDK in Windows Store application

The nineth update on 13/06/2016

Stable release 1.1.0:

• Add supporting of automating registration CaptureManager as COM server with Type Library
• Add supporting of dynamic language Python 2.7
• Add supporting of HEVC(H265) encoder in Windows 10: Download HEVCEncoder_Windows10

Capture Manager SDK 1.1.0 can be downloaded by link: Download CaptureManagerSDK-110.zip

The tenth update on 01/08/2016

Beta release 1.2.0:

• Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
• Lazy binding of Microsoft Media Foundation functions.
• Replace web camera properties functionality from DirectShow on DeviceIoControl. 
• Add resizing in EVR. 

Capture Manager SDK v1.2.0 beta can be downloaded by link: Download CaptureManagerSDK-1.2.0_beta.zip

The eleventh update on 05/09/2016:

Release 1.2.0:

• Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
• Lazy binding of Microsoft Media Foundation functions.
• Replace web camera properties functionality from DirectShow on DeviceIoControl. 
• Add resizing in EVR. 
• Add PULL mode in SampleGrabberCallSinkFactory - the new mode which allows to take a single sample.
• Add sample accumulator nodes for storing of 5 or 10 last samples in media stream.
• Add four new demo programms:

Capture Manager SDK v1.2.0 can be downloaded by link: Download CaptureManagerSDK-1.2.0-Freeware.zip

The twelveth update on 03/10/2016:

Beta release 1.3.0:

• Add supporting of video capturing via DirectShow Crossbar technique for the next inputs:                                  Composite,
       SVideo,
       USB,
       1394 (FireWire).

Capture Manager SDK v1.3.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware-beta.zip

The thirteenth update on 19/12/2016:

Add two new demo programms for Java programing language:

The fourteenth update on 02/01/2017:

Add NuGet CaptureManager SDK package - package for Net 4.0 projects. It is based on CaptureManagerToCSharpProxy, but there is a one difference - package includes CaptureManager.dll as embedded resource. It allows unpack CaptureManager.dll into the Temp folder at Runtime and upload it into the application process. It is more simple then register CaptureManager.dll as COM server module in system - it does not need admin privileges. This package supports both CaptureManager.dll for x86 and x64 arch CPU and select the suitable at Runtime.

The fifteenth update on 09/01/2017:

Release 1.3.0. This release includes the next changings:

• Add supporting of video capturing via DirectShow Crossbar technique for the next inputs:                                  Composite,
       SVideo,
       USB,
       1394 (FireWire);
• COM ThreadingModel has been changed from Apartment on Both.

Capture Manager SDK v1.3.0 can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware.zip

The sixteenth update on 16/01/2017:

Add the new demo program for on WindowsFroms GDI with supporting of source type selecting:

• WindowsFormsDemo

Beta release 1.4.0 - Version 1.4.0 Beta:

• Add supporting for changing of cursor presenting in Screen Capture source: make cursor invisible and add drawing additional image for current cursor's image.

Capture Manager SDK v1.4.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.4.0-Freeware-beta.zip

The seventeenth update on 06/02/2017:

Add the new demo programs:

Release 1.4.0 - Version 1.4.0. This release includes the next changings:

• Add supporting for changing of cursor presenting in Screen Capture source: make cursor invisible and add drawing additional image for current cursor's image.
• Add supporting for using of custom Microsoft Media Foundation Transform for implementation custom effects.

Capture Manager SDK v1.4.0 can be downloaded by link: Download CaptureManagerSDK-1.4.0-Freeware.zip

The eighteenth update on 13/03/2017:

Release 1.5.0 - Version 1.5.0. This release includes the next changings:

• Add supporting of multidisplay outputs.
• Add supporting of DirectShow software filters.

Capture Manager SDK v1.5.0 can be downloaded by link: Download CaptureManagerSDK-1.5.0-Freeware.zip

The nineteenth update on 01/05/2017:

Add the new demo programs:

Release 1.6.0 - Version 1.6.0. This release includes the next changings:

• Add IEVRMultiSinkFactory, IEVRStreamControl interfaces: IEVRMultiSinkFactory allows create two separated Topology Nodes for the one rendering contex - HWND of Window, Direct3DSurface9 or IDXGISwapChain. IEVRStreamControl allows control of rendering - position rendering, Z-order rendering, flushing of stream buffer, filters and features.
• IInitilaizeCaptureSource, ICurrentMediaType, ISourceRequestResult, ICaptureProcessor interfaces:  IInitilaizeCaptureSource receives configuration of Capture Processsor. ICurrentMediaType contains info of the selected Media Type. ISourceRequestResult receives raw data in format which is defined in IInitilaizeCaptureSource. ICaptureProcessor controls the capturing process.  

Capture Manager SDK v1.6.0 can be downloaded by link: Download CaptureManagerSDK-1.6.0-Freeware.zip

The twentyth update on 10/07/2017:

Add the new demo programs:

Release 1.7.0 - Version 1.7.0. This release includes the next changings:

• Add CaptureManager Video Renderer Factory which is compatable with MediaFoundation player.

Capture Manager SDK v1.7.0 can be downloaded by link: Download CaptureManagerSDK-1.7.0-Freeware.zip

The twenty-first update on 31/07/2017:

Add the new demo programs:

• Add UnityWebCamViewer demo program

The twenty-second update on 21/08/2017:

Add the new demo programs:

• Add WPFMultiSourceRecorder demo program.

The twenty-third update on 23/10/2017:

Release 1.8.0 - Version 1.8.0.  This release includes the next changings:

• Add "normalize" mode for the "Screen Capture" source.
• Add IRenderingControl interface for controlling of rendering on target process.  
• Add "Shape" attribute for "Cursor" option of "Screen Capture" source.
• Add "Clip" type option of "Screen Capture" source.

Add the new demo programs:

The twenty-fourth update on 15/01/2018:

Release 1.9.0 - Version 1.9.0.  This release includes the next changings:

• Add "HWND" mode for the "Screen Capture" source.

Add the new demo programs:

The twenty-fifth update on 05/03/2018:

Add the new demo programs:

• Add UnityVideoAndAudioRecorder demo program.

The twenty-sixth update on 09/04/2018:

Release 1.10.0 - Version 1.10.0.  This release includes the next changings:

• Add ISwitcherNodeFactory interface for pause/resume of the media stream.
• Add ISwitcherControl interface for controling (pause, resume, detach, attach) media streams.

Add the new demo programs:

The twenty-seventh update on 13/08/2018:

Release 1.11.0 - Version 1.11.0.  This release includes the next changings:

• Add DirectX11 Texture Capture.
• Change Screen Capture output format from RGB32 to NV12.

The twenty-eightth update on 01/10/2018:

Release 1.12.0 - Version 1.12.0.  This release includes the next changings:

• Add CM_DEVICE_LINK attribute with unique device id;
• Support DirectX9Ex texture shared handler.

Add the new demo programs:

The twenty-nineth update on 19/11/2018:

Release 1.13.0 - Version 1.13.0.  This release includes the next changings:

• Add supporting of DirectShow Video Renderer.

Add the new demo programs:

• Add DShowPlayer demo program;
• Add WPFDirectShowPlayer demo program;
• Add WPFRtspServer demo program.

The thirtyth update on 11/02/2019:

Release 1.14.0 - Version 1.14.0.  This release includes the next changings:

• Add H.264/MPEG-4 AVC video stream encoder (from 256 kbit/c to 64 kbit/c);
• Add H.265/MPEG-4 HEVC video stream encoder (64 kbit/c);
• Add AAC audio stream encoder (96 kbit/c);
• Add video stream decoder.

Add the new demo programs:

• Add WPFStreamer demo program;
• Add WPFRTSPClient demo program;
• Add WPFRtspServer demo program.

The thirty-first update on 15/04/2019:

Release 1.15.0 - Version 1.15.0.  This release includes the next changings:

• Add fast SSE2/SSSE3/SSE4.2 copy.

The thirty-second update on 06/07/2020:

Release 1.16.0 - Version 1.16.0.  This release includes the next changings:

• Add interfaces for mixering video and audio streams:

• IMixerNodeFactory,
• IVideoMixerControl,
• IAudioMixerControl.

Add the new demo programs:

• Add WPFMixer demo program;
• Add WPFGIFtoVideo demo program;
• Add AudioCaptureProcessor demo program;

The thirty-third update on 20/07/2020:

Publish source code of Capture Manager SDK under open source license. Source code can be downloaded by link: CaptureManagerSDK.zip

The thirty-fourth update on 16/11/2020:

Release 1.17.0 - Version 1.17.0.  This release includes the next changings:

• Expand H264 and H265 video encoder bitrate upto 2^25 (33,554,432)

The thirty-fifth update on 01/02/2021:

Release 1.18.0 - Version 1.18.0.  This release includes the next changings:

• Add Audio Renderer;
• Add WPFStreamingAudioRenderer demo program.

The thirty-sixth update on 12/04/2021:

Release 1.19.0 - Version 1.19.0.  This release includes the next changings:

• Add supporting of H264 webcam.

The thirty-seventh update on 06/09/2012:

Release 1.19.1 - Version 1.19.1.  This release includes the next changings:

• Fix bug with pull mode.
• Fix bug with open camera already in use.

The thirty-eighteenth update on 06/12/2012:

Release 1.20.0 - Version 1.20.0.  This release includes the next changings:

• Add async/await invoking model.

History

The first version was published on 11/08/2015.

The second version was published on 21/08/2015.

The third version was published on 31/08/2015.

The fifth version was published on 13/11/2015.

The sixth version was published on 07/03/2016.

The seventh version was published on 21/03/2016.

The eighth version was published on 12/04/2016.

The nineth version was published on 13/06/2016.

The tenth version was published on 01/08/2016.

The eleventh version was published on 05/09/2016.

The twelveth version was published on 03/10/2016.

The thirteenth version was published on 19/12/2016.

The fourteenth version was published on 02/01/2017.

The fifteenth version was published on 09/01/2017.

The sixteenth version was published on 16/01/2017.

The seventeenth version was published on 06/02/2017.

The eighteenth version was published on 13/03/2017.

The nineteenth version was published on 01/05/2017.

The twentyth version was published on 10/07/2017.

The twenty-first version was published on 31/07/2017.

The twenty-second version was published on 21/08/2017.

The twenty-third version was published on 23/10/2017.

The twenty-fourth version was published on 15/01/2018.

The twenty-fifth version was published on 05/03/2018.

The twenty-sixth version was published on 09/04/2018.

The twenty-seventh version was published on 13/08/2018.

The twenty-eightth version was published on 01/10/2018.

The twenty-nineth version was published on 19/11/2018.

The thirtyth version was published on 11/02/2019.

The thirty-first version was published on 15/04/2019.

The thirty-second version was published on 06/07/2020.

The thirty-third version was published on 20/07/2020.

The thirty-fourth version was published on 16/11/2020.

The thirty-fifth version was published on 01/02/2021.

The thirty-sixth version was published on 12/04/2021

The thirty-seventh version was published on 06/09/2021

The thirty-eighteenth version was published on 06/12/2021