Can AZCopy perform the copy starting from the most recent files and folders

0
votes

Hi all hope everyone is safe.

I'm migrating a Windows Application that contain 1.8 Million files and folders of images and other non-DB dependency files from our On-Prem DataCenter to an Azure VM. The application can tolerate missing images (it displays an X in their place), and I plan to use AZCopy to copy these files to the blob, however I noticed that AZCopy has its own way of choosing what files and folders to start with. Is there a way to let AZCopy start with the most recent files and folders? If that is possible, then I can do the cut-off as soon as AZCopy copies the last few days instead of waiting for the whole copy to be completed. So instead of few days downtime, it would be only few hours.

Thanks

1
azurecopyazure-powershellmigrateazcopy
Is that you want to upload files that were modified after a date and time? If so, please refer to docs.microsoft.com/en-us/azure/storage/common/… – Jim Xu
No Jim, eventually I want to move all the files, but I want the AZCopy to start with the most recent ones and go backwards in time to copy old files later. Keep in mind that the folder structure is immensely complicated where there are thousands of folders and subfolders, probably 4-5 levels. – Mohammad Taha
Hi. As far as I knew, we cannot use Azcopy to do that. According to the situation, I think you can copy the file you need at the moment to one folder then upload these. After doing that, you can upload other files by excluding the folder. – Jim Xu

1 Answers

0
votes

We have a way to filter to files after a point in time but not before.

Thank you for this suggestion. We are going add this in our backlog. Include-before shouldn't take too much work, since the foundation is already there.

As a suggestion in the meantime, and, this is not the most amazing workaround ever, but however it could potentially mark files after the time with a certain windows attribute that

  • Isn't on the other files
  • Won't affect your workload

Then, filter out the attribute. Unfortunately, all of the file attributes we can filter by do something.

// Available attributes (SMB) include:
// R = Read-only files
// A = Files ready for archiving
// S = System files
// H = Hidden files
// C = Compressed files
// N = Normal files
// E = Encrypted files
// T = Temporary files
// O = Offline files
// I = Non-indexed files
// Reference for File Attribute Constants:
// https://docs.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants