Kinect frame arrived asynchronously

Question

I am looking for some help with my MultiSourceFrameArrived event in the Kinect v2 SDK.

The following is the method:

private async void _reader_MultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e) { MultiSourceFrame multiSourceFrame = e.FrameReference.AcquireFrame(); using (var colorFrame = multiSourceFrame.ColorFrameReference.AcquireFrame()) { if (colorFrame != null) { _writeableBitmap.Lock(); colorFrame.CopyConvertedFrameDataToIntPtr( _writeableBitmap.BackBuffer, (uint)(_colorFrameDescription.Width * _colorFrameDescription.Height * _colorFrameDescription.BytesPerPixel), ColorImageFormat.Bgra); _writeableBitmap.AddDirtyRect(new Int32Rect(0, 0, _writeableBitmap.PixelWidth, _writeableBitmap.PixelHeight)); _writeableBitmap.Unlock(); reflectionOverlayControl.ReflectionImageSource = _writeableBitmap; } } using (var bodyFrame = multiSourceFrame.BodyFrameReference.AcquireFrame()) { if (bodyFrame != null) { Body body = JointHelpers.FindClosestBody(bodyFrame); if (body != null) { if (body.IsTracked) { Dictionary<BodyComponentType, BodyComponent> bodyComponentDictionary = BuildBodyComponentDictionary(body); foreach (BodyComponent bodyComponent in bodyComponentDictionary.Values.OrderBy(x => x.BodyComponentType)) { bodyComponent.Generate(_writeableBitmap, _coordinateMapper, FrameType.Color, 25); if (!_isRunningFiltering) { _isRunningFiltering = true; try { await Task.Run(() => { bodyComponent.RunFunFiltering(); }); } finally { _isRunningFiltering = false; } } } reflectionOverlayControl.UpdateValues( bodyComponentDictionary, GetFullBodyComponent(body)); } } } } } 

Now let me explain:

• The method is launched when a certain kind of frame comes from Kinect, it is received, and I can extract the ColorFrame and BodyFrame from it into use blocks.
• The first “used” block turns the ColorFrame into a WPF WriteableBitmap (declared in the constructor) and sets the ReflectionImageSource user control to this WriteableBitmap. If this was the only unit used, I would see a very smooth channel on the screen!
• The second BodyFrame method determines the closest body, if it is being tracked, and then creates a dictionary filled with people BodyComponents (arms, legs, head, etc.).
• Here, the foreach loop launches the Create function for each BodyComponent that sets some of its properties. For example, it sets the EncompassingRectangle property, which is an Int32Rect object designed to span a component.

The next bit is where I need help!

The RunFunFiltering method is a very intensive processing method that, when launched, will create a lock statement that freezes my user interface. This could cause my video frame with a color frame to be very nervous! This RunFunFiltering method should set some properties of the BodyComponent class, such as the color that the rectangle should display, the number of white pixels in it ReflectionImageSource, and set another recordable bitmap with the part of the first ReflectionImageSource contained in the rectangle.

Since this object is complete, all properties are set (and this was done for each of the BodyComponent in the dictionary), I run the UpdateValues ​​method in the view, which displays interesting material in the BodyComponent class on the screen for me.

Following @sstan's advice in this post: Async Await to continue triggering events

I ran the Task.Run () block. However, this does not seem to free my interface, and I still see a clumsy image. It is strange that in this timer example, it works fine! I lost a little here to know what to do.

I start a little with asynchronous functions, but I would really like to understand your solutions. If you can provide an explanation using your code, I would really appreciate it!

Update

I was able to determine that the using statement, which receives the frame, blocks the user interface when it is placed outside the Task.Run call.

I cannot just force the entire BodyFrame to use the block asynchronously, because I need the first Create function always and should not be part of a lot of processing. Two uses of blocks seem inelegant and rather push my question under the carpet ...

Answer 1

From your comment, I understand the following:

• You have an asynchronous call function that is called when the frame arrives
• If none of the RunFunFiltering tasks start, run
• If such a task is running, do not start a new one.
• If RunFunFiltering is Complete Process Result

.

 Task taskFunFiltering = null; private async Task ProcessFrame(...) { // a new frame is arrived DoSomeProcessing(...); // only start a new run fun filtering if previous one is finished if (taskFunFiltering == null || taskFunFiltering.IsCompleted) { // start a new fun filtering // don't wait for the result taskFunFiltering = Task.Run( () => ...); } } private async Task RunFunFiltering(...) { // do the filtering and wait until finished var filterResult = await DoFiltering(...); DisplayResult(filterResult); }