Changing Parallel.ForEach to better approach

Question

I use Task Parallel Library(TPL) and C# 4.5 to implement this business logic in a Windows Service Application:

Get JSON result (a list) from remote RESTful API
For each item, retrieve JSON result of details from another remote RESTful API
For each item's attached business objects(1000+), use Parallel.ForEach saving to database

Currently the problem is: It may take long time saving each item to database (Sync DAL), so Parallel.ForEach 1000 items takes forever and the Windows Service application appears slower and slower. Does anyone have good ideas or better approach to gain better performance?

Code snippet:

/* Download a list from RESTful API URL.... */
var task = Task.Factory.StartNew(() => { return DownloadListFromRestAPI(); }, TaskCreationOptions.LongRunning);
task.ContinueWith(i => {
     foreach (var r in i.Result)
      {
          /* For each item, download the item details from RESTful API URL.... */
           var taskSecond = Task.Factory.StartNew(() => { return DownloadItemDetailFromRestAPI(r.id); }, TaskCreationOptions.LongRunning);
           taskSecond.ContinueWith(m => {
               /* For each  item detail, get the related business objects, and start Database operation on each object.... */
                List<Item> relatedItems_1000 = s.GetRelatedObjectsIds(m.Result.id);

              /* parallel.ForEach - 1000 or more items  */
                 Parallel.ForEach<Item>(relatedItems_1000, new ParallelOptions { MaxDegreeOfParallelism = 8 }, d => DBLongProcess(d)); /* The DB operation takes long time */
          });
      }
 });

Update: (the code for DBLongProcess() and the lock (I added lock because concurrent threads may attempt to modify same object to DB))

 private void DBLongProcess(Item item)
 {
    dbDAL.InsertObjectDB(item));
 }

 public class DBDAL
 {
      private readonly object _lock = new object();

      public void InsertObjectDB(Item item)
      {
          lock (_lock)
          {
             if(!item.hasDetail1()){
                 //insert item.detail1...
             }
              if(!item.hasDetail2()){
                //insert item.detail2
             }
          }
      }
 }

How does DBLongProcess provide thread safety? If done wrong that easily could be the source of your slowness. Can you include the code for that too? — Scott Chamberlain
it is not clear what is the actual problem. the UI slows down because the Background takes too many resources? — Nahum
@NahumLitvin, the problem is the performance is bad. I have no UI and backgroundworker, it's a service application. My question is the line "parallel.foreach 1000 items" seems taking forever.... is there better approach to get better performance? — xoyoja

svick svick · Accepted Answer · 2013-12-25T12:49:43

In your case, using Parallel.ForEach() won't speed up anything, because the lock in InsertObjectDB() forces all items to be inserted in series.

What you need to do is to figure a way of making DBDAL thread-safe (possibly by using multiple instances of it). If that's not feasible, then you'll have to look for performance improvements elsewhere.

Changing Parallel.ForEach to better approach

1 Answers