AWS S3 CLI sync command free define time period configure

0
votes

I have .bat file inside that calls the AWS sync command:

aws s3 sync D:\Users\backup s3://mybucket

It syncs my local data to an S3 bucket. Then I created a Windows Scheduler Task for that .bat file, and every day at 2300hr the .bat file start to run and syncs my local data to the S3 bucket.

some time I have big data inside backup file. so i want to give particular time period to run aws s3 sync (example sync run only 1300hr to 0100hr)

note: if sync stop without all data sync next day should be resume sync at 1300hr

I have two questions:

  1. How can i force stop sync data given time period?

  2. If i have big data(more than 100GB) AWS CLI power enough to efficiently sync data ?

1
batch-fileamazon-s3aws-cli
What is your question/problem? Are you saying that the sync fails if there is too much data? Or are you saying that you want to force the sync to stop if it continues for more than a certain length of time? - John Rotenstein

1 Answers

0
votes

You can always kill the "rsync" task in Task Scheduler if it runs longer than 1,2,4,8 12 hours/days/etc.

Simply go to task properties in Task Scheduler and click on 'Settings" tab, you will see a checkbox "Stop the task if it runs longer than:" (select whatever value you want)

You would then be chipping away at your 100GB of files each night you run it.