FileUploadMiscError azure batch output file

Question

I am trying to upload output file to Azure blob after azure batch cloud task is done using c#. My code is working fine but it's giving FileUploadMiscError - Task failed "A miscellaneous error was encountered while uploading one of the output files" error randomly for different tasks.

ContainerSASUrl code -

 SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy
            {
                SharedAccessExpiryTime = DateTime.UtcNow.AddHours(sasUrlExpirationTime),
                Permissions = SharedAccessBlobPermissions.Write
            };
            string sasContainerToken = string.Empty;
            CloudBlobContainer container = BlobClient.GetContainerReference(containerName);

            if (await container.CreateIfNotExistsAsync())
            {
                sasContainerToken = container.GetSharedAccessSignature(sasConstraints);
            }
            sasContainerToken = container.GetSharedAccessSignature(sasConstraints);
       string outputContainerSasUrl= string.Format("{0}{1}", container.Uri, sasContainerToken);

Task submission -

 CloudTask task = new CloudTask(taskId, taskCommandLine)
                            {
                                OutputFiles = new List<OutputFile>
                                {
                                    new OutputFile(
                                        filePattern: @"../std*.txt",
                                        destination: new OutputFileDestination(
                                                        new OutputFileBlobContainerDestination(
                                                                containerUrl: outputContainerSasUrl,
                                                        path: jobdetails.JobId+"/output")),
                                        uploadOptions: new OutputFileUploadOptions(
                                                        uploadCondition: OutputFileUploadCondition.TaskCompletion)
                                    ),
                                    new OutputFile(
                                        filePattern: @"%AZ_BATCH_JOB_PREP_WORKING_DIR%\*.dat",
                                        destination: new OutputFileDestination(new OutputFileBlobContainerDestination(
                                                            containerUrl: outputContainerSasUrl,
                                                            path:jobdetails.JobId+"/output")),
                                        uploadOptions: new OutputFileUploadOptions(
                                                            uploadCondition: OutputFileUploadCondition.TaskCompletion))
                                }
                            };
                            task.UserIdentity = new UserIdentity(new AutoUserSpecification(AutoUserScope.Pool, ElevationLevel.Admin));
                            task.ResourceFiles = inputFiles;
                            TaskConstraints taskConstraints = new TaskConstraints();
                            taskConstraints.MaxTaskRetryCount = 2;
                            task.Constraints = taskConstraints;

This code is working fine, but it's failing randomly

Hydraxy Hydraxy · Accepted Answer · 2019-01-24T22:23:47

Can you check your fileuploadout.txt and fileuploaderr.txt which should be written alongside the task stdout.txt and stderr.txt? Ideally, if you can share them here (make sure to redact any details such as SAS's which are logged in them).

FileUploadMiscError azure batch output file

1 Answers