Performance decrease after each batch processed by a background job

Question

I noticed that my background job performing worse after each batch. Basically, job takes huge csv file and then updates relevant records in database in batches of 250 records. After each batch I am re-creating db context to keep ObjectContext tidy, but that doesn't help and it looks like just everything is getting slower after each batch. My job does:

Read 250 records from file
Create DbContext
Then for each record
Get corresponding entity from DB
Update properties (reflection)
When 250 records processed - SaveChanges

So when job starts, it takes ~170ms for 4. and ~6ms for 5.

After 50 batches, it takes ~430ms for 4. and ~500ms for 5.

Can someone explain please what is happening here?

Update. As it turned out it wasn't hangfire fault - so I removed all references to hangfire.

The problem seems to be with DbContext, despite I am recreating DbContext processing still takes longer after each batch. Also I missed one thing from 5. - call to dbContext.Entry(entity).Property("Guid").IsModified = false that takes longer each time:

//for each 250 items
var dbContext = new MyDbContext();

dbContext.Configuration.ValidateOnSaveEnabled = false;

foreach (var item in items)
{
    var entity = dbContext.MyItems
        .Include(x => x.SomeProp) // lots of these
        .FirstOrDefault(BuildFilterFrom(item)); //after each batch this taking longer (starting from 170ms and then after 50 batches it takes 430ms)

    //set entity propties from item
    //...

    dbContext.Entry(entity).Property("Guid").IsModified = false //after each batch this also taking longer (starting from 6ms and then after 50 batches it takes 500ms)
}
dbContext.SaveChanges();

Check the SQL script that gets generated by this process. I had something similar recently where Entity Framework created horrible SQL (not really its fault), and because I was executing it many many times, it was blowing up the query optimizer. Since you're not doing a bulk grab, that reduces the chances of my scenario applying, but I would still look at the database connection. — krillgar
What do system resources look like while this is running (potential memory leak)? — Speerian
@krillgar thanks for your comment, I think you might be right - it must be something wrong with by db calls.. — Vladimirs

krillgar krillgar · Accepted Answer · 2017-05-18T14:55:30

From your update, I think the best bet you would have is to ignore the Guid property when you change them. You should be able to add a .Where() to your collection of properties to eliminate them with that name. That way you save the microseconds of setting the property, and also of changing the IsModified property.

Performance decrease after each batch processed by a background job

2 Answers