Destination array is not long enough to copy all the items in the collection. Check array index and length

Question

I have this code. It gives me an error :

Destination array is not long enough to copy all the items in the collection. Check array index and length.

I thought that it was because of using a dictionary, so I switched it to ConcurrentDictionary, but the error is still here.

private void SaverCallback()
{
    AddThread("Main Thread");
    const string path = "milestone";
    while (!stop)
    {
        ConcurrentDictionary<string, object> milestone = new ConcurrentDictionary<string, object>();
        milestone.TryAdd("Jobs", JobQueue.Queue.MainQueue);
        milestone.TryAdd("Locked Jobs", JobQueue.Queue.LockedQueue);

    again: try {
            using (FileStream writingStream = new FileStream(path, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None))
            {
                BinaryFormatter formater = new BinaryFormatter();
                formater.Serialize(writingStream, milestone);
                writingStream.Flush();
                Logger.Debug("Status saved");
            }
        }
        catch(Exception e)
        {
            Logger.Error($"Milestone exception: {e.Message}");
            goto again;
        }
        this.WaitTime(60000);
    }
    RemoveThread();
}

UPD:

Destination array is not long enough to copy all the items in the collection. Check array index and length. and at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.Dictionary2.CopyTo(KeyValuePair2[] array, Int32 index) at System.Collections.Generic.Dictionary`2.GetObjectData(SerializationInfo info, StreamingContext context) at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter, SerializationBinder binder) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteArrayMember(WriteObjectInfo objectInfo, NameInfo arrayElemTypeNameInfo, Object data) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteArray(WriteObjectInfo objectInfo, NameInfo memberNameInfo, WriteObjectInfo memberObjectInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck)
at AggregateRunner.Enteties.Saver.SaverCallback()

don't use goto. it indicates a major design issue. the only languages where jumps are allowed are IL and ASM — Sebastian L
What is the type of JobQueue.Queue.MainQueue and JobQueue.Queue.LockedQueue? — Matthew Watson
So can another thread be adding items to the job queue while it is being serialised as part of the serialisation of milestone? If so, perhaps something bad is happening during that process. — Matthew Watson
so, if there is an exeption, log it and retry, using a goto, ad infinitum. Doesn't it sound bad? — Gian Paolo

VMAtm VMAtm · Accepted Answer · 2017-03-30T16:16:40

As far as I can see, you're taking a snapshot for your queues once an hour, but the main problem with your code is that you're trying to serialize your queue (which is a ConcurrentQueue<T>, I guess) itself without any cloning or synchronization logic. Another problem with your code is a goto usage, which is not necessary at all here.

The exception you're facing is a ArgumentException and is a third possible exception of CopyTo method of the ConcurrentQueue<T>. It's happening during the serialization because of some thread added some messages to queue and now it doesn't fit in array serializer decided to use.

So, what you can do in such situation is either introduce some locking during access to the original queues (which you're decided to do), or create a clone for your queues and serialize that in a safe manner without blocking other threads.

You can do this with CopyTo method of ConcurrentQueue<T> by yourself, with creating array with length more than number of messages in queue, but the locking in general is more convenient way to clone the data. So your code should be like this:

private void SaverCallback()
{
    AddThread("Main Thread");
    const string path = "milestone";
    while (!stop)
    {
        try
        {
            lock (JobQueue.Queue.MainQueue)
            lock (JobQueue.Queue.LockedQueue)
            {
                using (var writingStream = new FileStream(path, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None))
                {
                    var milestone = new ConcurrentDictionary<string, object>();
                    milestone.TryAdd("Jobs", JobQueue.Queue.MainQueue);
                    milestone.TryAdd("Locked Jobs", JobQueue.Queue.LockedQueue);

                    var formater = new BinaryFormatter();
                    formater.Serialize(writingStream, milestone);
                    writingStream.Flush();
                    Logger.Debug("Status saved");
                }
            }
            // this line cloud be in finally case too,
            // if you don't need to save a queue in case of some errors
            this.WaitTime(60000);
        }
        catch(Exception e)
        {
            // note that logger accepts whole exception information
            // instead of only it's message
            Logger.Error($"Milestone exception: {e}");
            continue;
        }
    }
    RemoveThread();
}

Destination array is not long enough to copy all the items in the collection. Check array index and length

2 Answers