Entity Framework inserting child objects in the wrong order

Question

Question

Why is EF first inserting a child object (PersonnelWorkRecord) with a dependency, before the object that it is depended on (TimesheetActivity). Also what are my options on correcting this?

ERD (simplified)

This is predefined by another system out of my direct control. ERD2

EF setup and save code

I am not sure I understand why/how Entity Framework is inserting the objects I have in the order it does however here is the code I am using to insert a parent and several children.

using (var db = new DataContext(user))
{
     timesheet.State = State.Added;
     timesheet.SequenceNumber = newSequenceNumber;
     this.PrepareAuditFields(timesheet);

     //To stop EF from trying to add all child objects remove them from the timehseets object.
     timesheet = RemoveChildObjects(timesheet, db);

     //Add the Timesheet object to the database context, and save.
     db.Timesheets.Add(timesheet);
     result = db.SaveChanges() > 0;
}

SQL Trace of EF's Inserts

When I run the code I get a SQL foreign key violation on the PersonnelWorkRecord (TimesheetActivityID) because I have not yet added the Activity (see trace).

exec sp_executesql N'insert [dbo].[Timesheets]([ProjectID], [TimesheetStatusID], ...
exec sp_executesql N'insert [dbo].[PersonnelWorkdays]([TimesheetID], [PersonnelID], ...
exec sp_executesql N'insert [dbo].[PersonnelWorkRecords]([PersonnelWorkdayID],[TimesheetActivityID], ...

Data Context Summary

modelBuilder.Entity<PersonnelWorkday>().HasRequired(pwd => pwd.Personnel).WithMany(p => p.PersonnelWorkdays).HasForeignKey(pwd => pwd.PersonnelID).WillCascadeOnDelete(false);
modelBuilder.Entity<PersonnelWorkday>().HasRequired(pwd => pwd.Timesheet).WithMany(t => t.PersonnelWorkdays).HasForeignKey(pwd => pwd.TimesheetID).WillCascadeOnDelete(false);
modelBuilder.Entity<PersonnelWorkRecord>().HasRequired(pwr => pwr.PersonnelWorkday).WithMany(pwd => pwd.PersonnelWorkRecords).HasForeignKey(pwr => pwr.PersonnelWorkdayID).WillCascadeOnDelete(false);
modelBuilder.Entity<PersonnelWorkRecord>().HasRequired(pwr => pwr.TimesheetActivity).WithMany(ta => ta.PersonnelWorkRecords).HasForeignKey(pwr => pwr.TimesheetActivityID).WillCascadeOnDelete(false);
modelBuilder.Entity<TimesheetActivity>().HasRequired(ta => ta.ProjectActivity).WithMany(a => a.TimesheetActivities).HasForeignKey(ta => ta.ProjectActivityCodeID).WillCascadeOnDelete(false);
modelBuilder.Entity<TimesheetActivity>().HasOptional(ta => ta.Facility).WithMany(f => f.TimesheetActivities).HasForeignKey(tf => tf.FacilityID).WillCascadeOnDelete(false);
modelBuilder.Entity<TimesheetActivity>().HasRequired(ta => ta.Timesheet).WithMany(t => t.TimesheetActivities).HasForeignKey(ta => ta.TimesheetID).WillCascadeOnDelete(false);

Remove Child Objects

Here is the code for the child objects method. I added this method to remove the objects from the timesheets' child objects related objects that are not foreign keys. For example I have a Crew object but I also have a CrewID foreign key, so I have set Crew = null so that EF does not try to insert it since it already exists.

private Timesheet RemoveChildObjects(Timesheet timesheet, DataContext db)
{
        timesheet.Crew = null;
        timesheet.Foreman = null;
        timesheet.Location = null;
        timesheet.Project = null;
        timesheet.SigningProjectManager = null;
        timesheet.TimesheetStatus = null;
        timesheet.Creator = null;
        timesheet.Modifier = null;

        if (timesheet.TimesheetActivities != null)
        {
            foreach (TimesheetActivity tsa in timesheet.TimesheetActivities)
            {
                tsa.Creator = null;
                if (tsa.EquipmentWorkRecords != null)
                {
                    tsa.EquipmentWorkRecords = RemoveChildObjects(tsa.EquipmentWorkRecords, db);
                }
                tsa.Facility = null;
                tsa.Modifier = null;
                if (tsa.PersonnelWorkRecords != null)
                {
                    tsa.PersonnelWorkRecords = RemoveChildObjects(tsa.PersonnelWorkRecords, db);
                }
                tsa.ProjectActivity = null;
                tsa.Structures = null;
                tsa.Timesheet = null;
            }
        }

        if (timesheet.TimesheetEquipment != null)
        {
            foreach (TimesheetEquipment te in timesheet.TimesheetEquipment)
            {
                te.Equipment = null;
                te.Timesheet = null;
            }
        }

        if (timesheet.EquipmentWorkdays != null)
        {
            timesheet.EquipmentWorkdays = RemoveChildObjects(timesheet.EquipmentWorkdays, true, db);
        }

        if (timesheet.TimesheetPersonnel != null)
        {
            foreach (TimesheetPersonnel tp in timesheet.TimesheetPersonnel)
            {
                tp.Personnel = null;
                tp.PersonnelWorkday = null;
                if (tp.PersonnelWorkday != null)
                {
                    tp.PersonnelWorkday = RemoveChildObjects(tp.PersonnelWorkday, db);
                }
                tp.Timesheet = null;
            }
        }

        if (timesheet.PersonnelWorkdays != null)
        {
            timesheet.PersonnelWorkdays = RemoveChildObjects(timesheet.PersonnelWorkdays, true, db);
        }

        return timesheet;
    }

Debug of values before EF save

From my understanding anything an dbContex.ObjectNameHere.Local will be added/modified/deleted when a dbContext.Save() is called. (Depending on what the entity State is set too.) Here is what EF is trying to save before I call the save() and get an SQL FK exception. Step1 Step2 Then I get the FK exception.

The INSERT statement conflicted with the FOREIGN KEY constraint "FK_PersonnelWorkRecords_TimesheetActivities". The conflict occurred in database "VPMTEST_GC", table "dbo.TimesheetActivities", column 'TimesheetActivityID'. The statement has been terminated.

Notes

Please let me know if there is anything I can post to help describe my question. I have looked around google / SO for answers but so far no solid answers, it looks like EF can not determine the order of inserting objects unless the Domain model is setup differently? I am not able to change the structure of most objects as they are used by another system. I can attempt to change my EF call, I would prefer not to use Raw SQL as the objects are quite a bit more extensive then the simplified versions I have posted here.

Similar questions: Self referencing entity and insert order

What does RemoveChildObjects remove exactly from the timesheet? You say only "child objects". Do you mean the timesheet.TimesheetActivities? In that case inserting new PersonnelWorkRecords would only work if their TimesheetActivityID refers to an already existing activity. But I'm probably misunderstanding... — Slauma
Thank you for the comment @Slauma, I have added the code for the child objects method. I can post the other levels if you think it would help. I did not clear the child object TimesheetActivites however I have cleared its reference child objects (aside from the FKs) so that EF would not try to insert them. Is this perhaps not the correct approach? — Gram
OK, I see. I think the approach is fine (attaching to the context instead of setting to null is the alternative, but it's neither better nor worse). Another question: You have 3 lines of SQL INSERT above, no insert line for the activities. Is that just the fourth line that you didn't show above or is an insert for the activities completely missing? (I guess it's the fourth line since you're talking about a wrong insert order, not about missing inserts. But I want to be sure...) — Slauma
@Slauma Good question, the trace ends after the 3rd line the activities never insert as the last line in the trace causes an exception (foreign key) for the TimehseetActivityID value. So there is missing inserts, I assume that the values would be inserted later if an exception was not thrown. I assume this because the Entry.TimehseetActivity.Local has values in it when I view it before I use the db.Save(). (I will post the debugger screen.) — Gram
Ah, of course, the exception stops the insertion sequence, I could have thought of this myself. I've tried an answer in the meantime, see below, however it contains lots of guesses. — Slauma

Slauma Slauma · Accepted Answer · 2014-02-19T21:16:53

In your RemoveChildObjects method I see the line...

tsa.Timesheet = null;

So, apparently your are setting the inverse navigation property of Timesheet.TimesheetActivities to null. Are you doing the same with PersonnelWorkRecord.TimesheetActivity and PersonnelWorkRecord.PersonnelWorkday, i.e. do you set those properties to null as well in the nested RemoveChildObjects methods?

This could be a problem because you have two different paths from Timesheet to PersonnelWorkRecord, namely:

Timesheet -> TimesheetActivities -> PersonnelWorkRecords
Timesheet -> PersonnelWorkdays -> PersonnelWorkRecords

When you call db.Timesheets.Add(timesheet) I believe EF will traverse each branch in the object graph one by one and determine on the path which related objects ("nodes") are dependent and which are principal in a relationship to determine the order of insertion. timesheet itself is principal for all its relationships, therefore it is clear that it must be inserted first. Then EF starts to iterate through one of the collections Timesheet.TimesheetActivities or Timesheet.PersonnelWorkdays. Which one comes first doesn't matter. Apparently EF starts with Timesheet.PersonnelWorkdays. (It would not solve the problem if it would start with Timesheet.TimesheetActivities, you would get the same exception, but with PersonnelWorkRecord.PersonnelWorkday instead of PersonnelWorkRecord.TimesheetActivity.) PersonnelWorkday is only dependent on Timesheet which is already inserted. So, PersonnelWorkday can be inserted as well.

Then EF continues traversing with PersonnelWorkday.PersonnelWorkRecords. With respect to the PersonnelWorkday dependency of PersonnelWorkRecord there is again no problem because the PersonnelWorkday has already been inserted before. But when EF encounters the TimesheetActivity dependency of PersonnelWorkRecord it will see that this TimesheetActivity is null (because you've set it to null). It assumes now that the dependency is described by the foreign key property TimesheetActivityID alone which must refer to an existing record. It inserts the PersonnelWorkRecord and this violates a foreign key constraint.

If PersonnelWorkRecord.TimesheetActivity is not null EF would detect that this object hasn't been inserted yet but it is the principal for PersonnelWorkRecord. So, it can determine that this TimesheetActivity must be inserted before the PersonnelWorkRecord.

I would hope that your code works if you don't set the inverse navigation properties to null - or at least not the two navigation properties in PersonnelWorkRecord. (Setting the other navigation properties like tsa.Creator, tsa.Facility, etc. to null should not be a problem because those related objects really already exist in the database and you have set the correct FK property values for those.)