I'm trying to import city data from a csv file, some data may be duplicated which invoke conflict error ERROR: duplicate key value violates unique constraint "city__unique_idx"
Detail: Key (country, city_name, latitude, longitude)=(231, Monticello, 30.5450000000000017, -83.8703000000000003) already exists.
2018-03-13 14:34:03.607 ERROR 13275 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step cityStep1 in job importCityJob
. I want to know how to ignore this error and keep the job running because it currently exits immediately.
Writer:
@Bean
public ItemWriter<City> cityWriter(EntityManagerFactory factory) {
JpaItemWriter<City> writer = new JpaItemWriter<>();
writer.setEntityManagerFactory(factory);
return writer;
}
Here is my job method:
@Bean
public Step cityStep1(ItemWriter<City> writer) {
return stepBuilderFactory.get("cityStep1").<City, City>chunk(10).reader(cityReader())
.processor(cityProcessor()).writer(writer).build();
}
@Bean
public Job importHotelJob(JobCompletionNotificationListener listener, Step cityStep) {
return jobBuilderFactory.get("importCityJob").incrementer(new RunIdIncrementer())
.listener(listener)
.next(cityStep1)
.build();
Thanks.
Edit1:
After I applied faultTolerant()
@Bean
public Step cityStep1(ItemWriter<City> writer) {
return stepBuilderFactory.get("cityStep1").<City, City>chunk(50).reader(cityReader())
.processor(cityProcessor()).writer(writer)
.faultTolerant()
.skip(ConflictException.class)
.skip(ConstraintViolationException.class)
.noRetry(ConflictException.class)
.noRetry(ConstraintViolationException.class)
.skipLimit(150)
.build();
I still get error:
2018-03-14 15:49:40.047 ERROR 26613 --- [ main] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: duplicate key value violates unique constraint "city__unique_idx"
Detail: Key (country, city_name, latitude, longitude)=(231, Monticello, 30.5450000000000017, -83.8703000000000003) already exists.
2018-03-14 15:49:40.161 ERROR 26613 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step cityStep1 in job importCityJob
org.springframework.retry.ExhaustedRetryException: Retry exhausted after last attempt in recovery path, but exception is not skippable.; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$5.recover(FaultTolerantChunkProcessor.java:405) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:512) ~[spring-retry-1.2.1.RELEASE.jar:na]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:351) ~[spring-retry-1.2.1.RELEASE.jar:na]
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211) ~[spring-retry-1.2.1.RELEASE.jar:na]
150
for bad records is breached. – Sabir Khan