I am using Java SDK to manage my S3 buckets. At this point in time, I want to introduce a new use case for this bucket i.e. I want to store similar objects but with a different retention period. I learnt that I can set lifecycle policies by prefix. I want to use this convenience provided by S3 to have a different lifecycle configuration for the new objects I am storing in my bucket identifiable by a different prefix.
Here is my Java program that does this:
package com.something.aws.s3;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.BucketLifecycleConfiguration;
import com.amazonaws.services.s3.model.lifecycle.LifecycleFilter;
import com.amazonaws.services.s3.model.lifecycle.LifecyclePrefixPredicate;
import org.junit.Before;
import org.junit.Test;
import java.util.List;
public class S3ClientTest {
private AmazonS3 s3;
@Before
public void setup() {
s3 = AmazonS3ClientBuilder.defaultClient();
}
@Test
public void test_getBuckets() {
for (Bucket b : s3.listBuckets()) {
System.out.println(b.getName());
}
}
@Test
public void test_addPrefixRules() {
String bucketName = "test-bucket";
BucketLifecycleConfiguration.Rule rule = new BucketLifecycleConfiguration.Rule()
.withId("new-lifecycle-id")
.withExpirationInDays(1)
.withFilter(new LifecycleFilter(new LifecyclePrefixPredicate("remote/")));
List<BucketLifecycleConfiguration.Rule> bucketLifecycleConfigurationRules =
s3.getBucketLifecycleConfiguration(bucketName).getRules();
bucketLifecycleConfigurationRules.add(rule.withStatus(BucketLifecycleConfiguration.ENABLED));
BucketLifecycleConfiguration updatedBucketLifecycleConfiguration =
new BucketLifecycleConfiguration().withRules(bucketLifecycleConfigurationRules);
s3.setBucketLifecycleConfiguration(bucketName, updatedBucketLifecycleConfiguration);
}
}
My test test_addPrefixRules tries to fetch all lifecycle config rules defined for a bucket, add my new rule for the prefixed objects and update the bucket with the new set of rules.
When I run this, I run into the following exception:
com.amazonaws.services.s3.model.AmazonS3Exception: Filter element can only be used in Lifecycle V2. (Service: Amazon S3; Status Code: 400; Error Code: InvalidRequest; Request ID: 6E24BE84B13B6440; S3 Extended Request ID: Y5VyIEuXcbGVBoddgdT7TVVa670dU6V1v5Oizlq/UU2pSDNjiAxyzrc155uf0MTfZxPw1GyUvGM=)
, S3 Extended Request ID: Y5VyIEuXcbGVBoddgdT7TVVa670dU6V1v5Oizlq/UU2pSDNjiAxyzrc155uf0MTfZxPw1GyUvGM=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4325)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4272)
at com.amazonaws.services.s3.AmazonS3Client.setBucketLifecycleConfiguration(AmazonS3Client.java:2326)
at com.amazonaws.services.s3.AmazonS3Client.setBucketLifecycleConfiguration(AmazonS3Client.java:2293)
at com.appdynamics.eum.aws.s3.S3ClientTest.test_addPrefixRules(S3ClientTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
I found very little documentation and discussion in forums that can explain this error. Can someone help me out in understanding what I might be doing wrong here?
