Discard certain items in the data stream pipeline

Question

I am creating a Dataflows pipeline whose task is to process large files. Each file is analyzed, analyzed and visualized; but each file can go a different path along the pipeline, depending on what type of file it has.

The user interface for this pipeline consists of a list of processed files, as well as a progress bar and a Cancel button next to each file (and, of course, a button for adding a new file to the queue). When a user clicks the Cancel button next to a specific file, I would like to remove only one file from the pipeline.

Something must be missing for me, because I cannot figure out how to do it. I know that I can undo the whole block, but I do not want to do this, I just want to undo one element in the pipeline. So what am I missing?

Answer 1

TPL Dataflow does not support canceling certain items out of the box.

You can implement this yourself by creating a wrapper over an element using the appropriate one CancellationTokenand placing it in the pipeline, and not just in the file. Then simply add code to each block that ignores this file if the token has been canceled and the element will pass quickly:

var block = new ActionBlock<FileWrapper>(wrapper => 
{
    if (wrapper.CancellationToken.IsCancelltionRequested)
    {
        return;
    }

    ProcessFile(wrapper.File);
});

This means that you have one token per item, which allows you to target individual items.