Datagridview virtualmode with heavy data load

Question

I want to display a "lot of heavy data" inside datagridview in virtual mode.

I use a buffer where I store n (2) pages of data, initially 2 first pages are stored in buffer. When the gridview is scrolled upto a row that does not exist in buffer i load the current new page and swap it with an old page.

To simulate loading heavy data from database into the buffer, I added this line: System.Threading.Thread.Sleep(3000);
as the result , as the buffer update itself the grid is frozen for this period of time.

Now, as I understand adding a thread won't help here because there is only 1 operation that happens only when scroll reaches unbuffered row.

  private void dataGridView1_CellValueNeeded(object sender, DataGridViewCellValueEventArgs e)
    {

       e.Value = theBuffer.ReturnFromBuffer(e.RowIndex, e.ColumnIndex);
//I check there if row index is in buffer and if not i update the buffer with new page and return the value

    }

So I am thinking about adding a "spinning wheel" animation above the gird when it happens.

I tried, to send event to form to show/hide the image within the method that updates the buffer:

if (ProgressEvent != null)
     ProgressEvent(true);

System.Threading.Thread.Sleep(3000);

if (ProgressEvent != null)
     ProgressEvent(false);

and on form event handler set the visible property of the image, but the result was not as expected, i guess Sleep happened before event was handled, so the image didn't appear before the sleep.

Som I guess that a tread is still needed. But can't figure when it sholud started and what should it do?

Update:

I tired the suggested background worker:
Inside the method that is beeing called by CellValueNeeded event, i check if the requested row is buffered, if not i start backgroundworker.

 if (!bgWorker.IsBusy) //otherwise worker will be started for every column of the row
 {
      bgWorker.DoWork += (sender, e) =>
         {
           UpdateBuffer(rowIndex);                           
          };
       bgWorker.RunWorkerAsync();
 }
 return "null";

And this:

bgWorker.RunWorkerCompleted += (sender, e) =>
{
    if (IvalidateEvent != null)
        IvalidateEvent();  //send event to form, where invalidate the gridview  dataGridView1.Invalidate();
};

it works but there a few problems with that: 1) I got many dummy results in my grid, even if it's not for long - I don't like it much. 2)it even takes more time if I just left it as is, probably because backgroundworker consumes some time and because I actually invalidate grid and call CellValueNeeded event 2 times (first onem to start backgroundworkerm second time when it finishes it's work)!

invalidating the whole grid is a bit like "overkill" ... it will also repaint all cells that have correct values, instead of only those with dummies. another approach to avoid dummy values while scrolling through the grid would be to react on the scrolling itself. try to fetch rows that are close to the displayed rows before they are requested, in that case the values will likely be there before CellNeedsValue requests them — DarkSquirrel42
@DarkSquirrel42, CellNeedsValue buffers not only the needed row but a bunch of rows ahead. the only problem when I need to load new rows. I don't mind to freeze the grid, but animation must appear as it happens. without dummy values — Mark A
when the datagridview raises cellvalueneeded you can specify a value to be shown in the cell. if you don't have that value, you can't specify it. so you have 2 options: 1) either provide a "loading"-message or nothing (empty string) , fill your buffer in an async operation and when the data is ready update the cell. or option 2) fill the buffer synchronously and return the desired value. of course option 2 will block the UI thread, so no loading animation, form repainting, or user interaction will be available (unless you do bad things like Application.DoEvents - that will be a pain to debug) — DarkSquirrel42
the problem with that is when I,for example, click on pagedown many times, so dummy value fills many rows (until data is loaded). for testing I use dataset as source and datatable that holds page in memory. Maybe other resources would work faster? — Mark A
i don't think so ... try optimizing the loader process ... create a class that handles requests to the buffer that require to load data ... store such requests and when your worker thread runs, try to collect more than just one row (reduce the number of queries) ... when the worker has finished his job, have it look at the list of requests and continue using that worker instead of spawning new workers if ther's more to do... track the scroll event and make sure that the next and prevous X rows are in the buffer (so after scrolling the buffer can supply the value directly) — DarkSquirrel42

DarkSquirrel42 DarkSquirrel42 · Accepted Answer · 2011-04-21T13:38:24

when you reach an unbuffered line, call your code to get the required data in a 2nd thread (new Thread, threadpool, backgroundworker ... choose one)

after this async operation has been started make the "loading..." control visible

(here you will probably want to disable parts of your UI until the operation is completed)

for now return an empty string or some other dummy value...

your async operation should get the desired data and then invoke another method the ui thread (call Invoke(...) on your Form's object)

the invoke call is needed to avoid cross-thread UI interaction

in the invoked method you should use the fetched data to populate your buffer, hide the "loading..." control, invalidate the corresponding row(s) or cell(s) (the datagridview has methods to do this) and last but not least reenable your UI if necessary

Datagridview virtualmode with heavy data load

1 Answers