background worker process and filesystemwatcher error working together

Question

Hi I have a problem using Filesystemwatcher & BackgroundWorker process.

I have a windows forms application that checks for new text files on a folder, it process them and creates xml files from them.

I´m using FSW to monitor for new txt files on a folder, the app works fine but when the folder receives a large amount of files (let's say 1000), the app freezes because it's processing all of them.

it occurred to me to add a backgroundworker, so the FSW calls it everytime a new file is created, this way we can process the file on the background without freezing the UI.

This idea did not work because for every file that is created, I try to call the RunWorkerAsync() method, so if it's busy processing a file and I try to process a new one it will throw the following error:

"This BackgroundWorker is currently busy and cannot run multiple tasks concurrently."

So I tried to loop the method with a while til it gets available but, infinite exception is thrown. this is the simplified version of my code:

private void fileSystemWatcher1_Created(object sender, System.IO.FileSystemEventArgs e)
    {
        readFile();

    }

    private void readFile()
    {
        while (backgroundWorker1.IsBusy)
        {
            readFile();
        }
        backgroundWorker1.RunWorkerAsync(idx);

    }

        private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
    {
        int i = (int)e.Argument;
        i += 1;
        e.Result = i;
    }

    private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
    {
        label1.Text = "Processing...";
        this.Refresh();
    }

    private void backgroundWorker1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
    {
        label1.Text = "Completed...";
        this.Refresh();
        idx = (int)e.Result;
    }

The exception thrown says “An unhandled exception of type 'System.StackOverflowException' occurred in WindowsFormsApplication2.exe, make sure you do not have an infinite loop or recursion”

Of course I could remove the FSW, but I'd like to know if there's a way to make them work together, any ideas?

Answer 1

What you have is a classic Producer/Consumer problem.

Solve it with a System.Collections.Concurrent.ConcurrentQueue<string>.

• On the FSW event, add the filename to the Queue.
• Start 1 or 2 BackgroundWorkers to process the queue.

---

And this is the code that overflows your stack in a hurry:

private void readFile()
{
    while (backgroundWorker1.IsBusy)
    {
        readFile();  // the recursive call, will fail quickly
    }
    backgroundWorker1.RunWorkerAsync(idx);
}

Not only does this cause an SO exception, it also blocks your main thread.
You need a better way to wait, and the ConcurrentQueue gives you that.

Answer 2

Instantiating new BackgroundWorkers would do the trick, as would Henk's solution above.

Or, you can do it without changing your code too much just using the ThreadPool.

    private void fileSystemWatcher1_Created(object sender, System.IO.FileSystemEventArgs e)
    {
        ThreadPool.QueueUserWorkItem(o => readFile(e));
    }


    public void readFile(System.IO.FileSystemEventArgs e)
    {
        this.BeginInvoke(new MethodInvoker(() =>
                                               {
                                                   label1.Text = "Processing...";
                                                   this.Refresh(); //you shouldn't need this
                                               }));

        //your long running read/processing... doing something event args

        this.BeginInvoke(new MethodInvoker(() =>
                                               {
                                                   label1.Text = "Completed...";
                                                   this.Refresh();
                                                   idx = (int) e.Result;
                                               }));
    }

Answer 3

Why not instantiate a new BackgroundWorker in readFile instead of reusing?