Binary Formatter Custom Binding for Nested Class

Question

So I'm writing a migration tool to convert a bunch of data files that are just binary formatted objects to our nice new YAML format. The problem is, we've also made changes to the underlying class since then. So my plan is to deserialise the on-disk data using a renamed version of the class taken from source control, map the properties over to the new version and then serialise the new one to YAML.

So the "old" class is this:

     [Serializable]
        public class OldQuestionnaire : INotifyPropertyChanged
        {
            private static Random random = new Random((int)DateTime.Now.Ticks);
            private bool synchronised;
    
            public OldQuestionnaire()
            {
                Questions = new List<Question>();
                QuestionnaireInstanceID = random.Next(0, int.MaxValue);
            }
    
            [field: NonSerialized]
            public event PropertyChangedEventHandler PropertyChanged;
    
            public string Customer { get; set; }
            public DateTime FilledDate { get; set; }
            public string FSR { get; set; }
            public bool IsPreferred { get; set; }
            public bool IsSiteSpecific { get; set; }
            public string JobRef { get; set; }
            public int QuestionnaireInstanceID { get; }
            public string QuestionnaireTemplateID { get; set; }
    
            //Database template ID
            public List<Question> Questions { get; set; }
    
            public DateTime ReleaseDate { get; set; }
            public string Site { get; set; }
    
            public bool Synchronised
            {
                get => synchronised;
                set
                {
                    synchronised = value;
                    OnPropertyChanged();
                }
            }
    
            public string TempStorageFilePath { get; set; }
            public string Title { get; set; }
    
            public void DeleteStoredCopy()
            {
                if (TempStorageFilePath == null || !File.Exists(TempStorageFilePath))
                {
                    return;
                }
                File.Delete(TempStorageFilePath);
            }
    
            public OldQuestionnaire GetBlankCopy()
            {
                OldQuestionnaire copy = new OldQuestionnaire
                {
                    Questions = Questions.Select(q => q.Clone()).Cast<Question>().ToList(),
                    Title = Title,
                    ReleaseDate = ReleaseDate,
                    QuestionnaireTemplateID = QuestionnaireTemplateID
                };
                return copy;
            }
    
            public void StoreToDisk(string tempPath)
            {
                string fullPath = Path.Combine(tempPath, $"{QuestionnaireInstanceID}.csat");
                TempStorageFilePath = fullPath;
                if (File.Exists(fullPath))
                {
                    File.Delete(fullPath);
                }
                BinaryFormatter formatter = new BinaryFormatter();
                using (Stream fStream = File.OpenWrite(fullPath))
                {
                    formatter.Serialize(fStream, this);
                }
            }
    
            protected void OnPropertyChanged([CallerMemberName] string propertyName = null)
            {
                PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
            }
        }

Note, it used to just be called Questionnaire.

So the migration process ought to be as simple as:

            OldQuestionnaire oldQ;
            using (StreamReader reader = new StreamReader("../../../libcsatmigrations/BinaryToYAML1/ReferenceFiles/1753324844.csat"))
            {
                BinaryFormatter formatter = new BinaryFormatter {Binder = new OldQBinder()};
                oldQ = (OldQuestionnaire)formatter.Deserialize(reader.BaseStream);
            }
            BinaryToYAMLMigrator migrator = new BinaryToYAMLMigrator();
            string yaml = migrator.Migrate(oldQ);

Where migrator.Migrate(oldQ) is handling the property mapping and such.

Initially, this raised issues with the BinaryFormatter not finding the needed assembly for OldQuestionnaire, so I created a custom binder:

    public class OldQBinder:SerializationBinder
    {
        public override Type BindToType(string assemblyName, string typeName)
        {
            return typeName switch
                   {
                       "libcsatquestionnaire.Question" => typeof(Question),
                       "libcsatquestionnaire.Questionnaire" => typeof(OldQuestionnaire),
                       "libcsatquestionnaire.OneTenQuestion" => typeof(OneTenQuestion),
                       "libcsatquestionnaire.FreeCommentQuestion" => typeof(FreeCommentQuestion),
                        _ => null,
                   };
        }
    
    }

(As a side note, loving the new C#8 pattern stuff)

This seems to have fixed that but now I'm getting a pretty similar issue with it not being able to find the nested custom type, so I'd imagine the binder is only called once.

System.Runtime.Serialization.SerializationException: Unable to load type System.Collections.Generic.List`1[[libcsatquestionnaire.Question, libcsatques...

System.Runtime.Serialization.SerializationException Unable to load type System.Collections.Generic.List`1[[libcsatquestionnaire.Question, libcsatquestionnaire, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]] required for deserialization. at System.Runtime.Serialization.ObjectManager.CompleteObject(ObjectHolder holder, Boolean bObjectFullyComplete) at System.Runtime.Serialization.ObjectManager.DoNewlyRegisteredObjectFixups(ObjectHolder holder) at System.Runtime.Serialization.ObjectManager.RegisterObject(Object obj, Int64 objectID, SerializationInfo info, Int64 idOfContainingObj, MemberInfo member, Int32[] arrayIndex) at System.Runtime.Serialization.Formatters.Binary.ObjectReader.RegisterObject(Object obj, ParseRecord pr, ParseRecord objectPr, Boolean bIsString) at System.Runtime.Serialization.Formatters.Binary.ObjectReader.RegisterObject(Object obj, ParseRecord pr, ParseRecord objectPr) at System.Runtime.Serialization.Formatters.Binary.ObjectReader.ParseObjectEnd(ParseRecord pr) at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Parse(ParseRecord pr) at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, IMethodCallMessage methodCallMessage) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream) at libcsatmigrations_tests.BinaryToYAMLMigratorTests.CanMigrateToYAML() in C:\Users\James H\Documents\repos\CSAT-Tooling\libcsatmigrations_tests\BinaryToYAMLMigratorTests.cs:line 21

So, my question is thus, how do I create a binder to deserialise these nested types?

Answer 1

Note: This works but isn't a solution to the underlying problem, more a possible workaround that may help someone else. If someone has an actual answer, please do post it.

---

I recloned the repo, reset the head to commit before the last release to get all of the correct assemblies and pointed it at a new upstream. Sadly, this won't be able to be integrated into the original project.

From here, I'm deserialising the old, Binary Formatted data and reserialising it as is in YAML.

            string              path           = Path.Combine(Environment.ExpandEnvironmentVariables("%appdata%"), "SE-CSAT","Stored");
            //Backup old files
            ZipFile backup = new ZipFile();
            backup.AddDirectory(path);
            backup.Save(new FileStream(path.Replace("\\Stored","backup.zip"),FileMode.Create));
            //Read and deserialise old files
            List<string>        files     = Directory.GetFiles(path,"*.csat").ToList();
            BinaryFormatter     formatter = new BinaryFormatter();
            List<Questionnaire> questionnaires = new List<Questionnaire>();
            foreach (string file in files)
            {
                using (FileStream fStream = new FileStream(file, FileMode.Open))
                {
                    questionnaires.Add((Questionnaire)formatter.Deserialize(fStream));
                }
            }
            ISerializer serializer = new SerializerBuilder()                           
                                    .WithTagMapping("tag:yaml.org,2002:OneTenQuestion",typeof(OneTenQuestion))
                                    .WithTagMapping("tag:yaml.org,2002:FreeCommentQuestion",typeof(FreeCommentQuestion))
                                    .Build();
            //Delete old files
            files.ForEach(File.Delete);
            //Reserialise as YAML
            List<string> yaml = questionnaires.Select(serializer.Serialize).ToList();
            //Write it all to disk
            string       newName;
            int          i = 0;
            foreach (string data in yaml)
            {
                do
                {
                    newName = Path.Combine(path, $"converted-{i++}.yml");
                } while (File.Exists(newName));
                File.WriteAllText(newName, data);
            }

I'll then package this executable with my update package (which uses the new assemblies) and run it at the start of the upgrade process. Migrating YAML is then a fairly straightforward mapping procedure.

In order to prevent assembly clashes, I'll use Fody/Costura to embed the needed assemblies for this small executable into the binary.

---

Update: I did try compiling this to a library with Costura too in the hopes that the weaving would prevent assembly conflicts when called from other projects. Sadly, that didn't work so I'm stuck with calling this executable.

I did add some output to STDOUT so that I can redirect the output to a stream in the calling application and establish what's going on at runtime.