How Do You "Really" Serialize Circular Referencing Objects With Newtonsoft.Json?

Question

I'm having a problem getting some data serialized correctly from my ASP.NET Web API controller using Newtonsoft.Json.

Here's what I think is going on - please correct me if I'm wrong. Under certain circumstances (specifically when there aren't any circular references in the data) everything works just like you'd expect - a list of populated objects gets serialized and returned. If I introduce data that causes a circular reference in the model (described below, and even with PreserveReferencesHandling.Objects set) only the elements of the list leading up to the first object with a circular reference get serialized in a way that the client can "work with". The "elements leading up to" can be any of the elements in the data if it's ordered differently before sending things to the serializer, but at least one will be serialized in a way the client can "work with". The empty objects end up being serialized as Newtonsoft references ({$ref:X}).

For example, if I have an EF model complete with navigation properties that looks like this:

In my global.asax:

var json = GlobalConfiguration.Configuration.Formatters.JsonFormatter;
json.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;

Here's the fundamental query I'm doing using Entity Framework (lazy-loading is off so there aren't any proxy classes here):

[HttpGet]
[Route("starting")]
public IEnumerable<Balance> GetStartingBalances()
{
   using (MyContext db = new MyContext())
   {
       var data = db.Balances
        .Include(x => x.Source)
        .Include(x => x.Place)
        .ToList()
       return data;
    }
}

So far so good, data is populated.

If there are no circular references, life is grand. However, as soon as there are 2 Balance entities with the same Source or Place, then the serialization turns the later Balance objects of the top-most list that I'm returning into Newtonsoft references instead of their full-fledged objects because they were already serialized in the Balances property of the Source or Place object(s):

[{"$id":"1","BalanceID":4,"SourceID":2,"PlaceID":2 ...Omitted for clarity...},{"$ref":"4"}]

The problem with this is that the client doesn't know what to do with {$ref:4} even though we humans understand what's going on. In my case, this means that I cannot use AngularJS to ng-repeat over my entire list of Balances with this JSON, because they aren't all true Balance objects with a Balance property to bind. I'm sure there are tons of other use-cases that would have the same problem.

I can't turn off the json.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects because lots of other things would break (which is well-documented in 100 other questions here and elsewhere).

Is there a better workaround for this apart from going through the entities in the Web API controller and doing

Balance.Source.Balances = null;

to all of the navigation properties to break the circular references? Because THAT doesn't seem right either.

Answer 1

Yes, using PreserveReferencesHandling.Objects is really the best way to serialize an object graph with circular references, because it produces the most compact JSON and it actually preserves the reference structure of the object graph. That is, when you deserialize the JSON back to objects (using a library that understands the $id and $ref notation), each reference to a particular object will point to the same instance of that object, rather than having multiple instances with the same data.

In your case the problem is that your client side parser does not understand the $id and $ref notation produced by Json.Net, so the references are not being resolved. This can be fixed by using a javascript method to reconstruct the object references after deserializing the JSON. See here and here for examples.

Another possibility which might work, depending on your situation, is to set ReferenceLoopHandling to Ignore when serializing instead of setting PreserveReferencesHandling to Objects. This is not a perfect solution though. See this question for a detailed explanation of the differences between using ReferenceLoopHandling.Ignore and PreserveReferencesHandling.Objects.

Answer 2

I wrote a minimal program to test this. Here is my github: https://github.com/assafwo1/TestSerializeJsonObjects. Here is the code:

using Newtonsoft.Json;
using System.Diagnostics;

namespace TestSerializeJsonObjects
{
    class Program
    {
        public class Node
        {
            public Node Next { get; set; }
        }
        static void Main(string[] args)
        {
            // create new node
            var head = new Node();
            // point its "next" field at itself
            head.Next = head;
            // this is now the smallest circular reference data structure possible 
            // assert that head next is head
            Debug.Assert(head.Next == head);
            // save to string
            var s = JsonConvert.SerializeObject(head, new JsonSerializerSettings
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects
            });
            // get from string
            var head2 = JsonConvert.DeserializeObject<Node>(s, new JsonSerializerSettings
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects
            });
            // assert that head2 next is head2
            Debug.Assert(head2.Next == head2);
            // done
        }
    }
}