Implementing SkipListener to write invalid records to a flat file

Question

I am working on setting a spring batch job that does the conventional READ > PROCESS > WRITE operation. However, I am trying to implement a listener which would capture records that are conisdered invalid during the PROCESS phase and write it out to an error log file.

My listener class uses an instance of FlatFileItemWriter to write the data. However, spring-batch is not instantiating the writer instance properly.

My listener class looks like this:

    public class DTOProcessorListener extends SkipListenerSupport<AttributeReportGenerationDTO, AttributeValue> {

    private static final Logger LOGGER = LoggerFactory.getLogger(DTOProcessorListener.class);

    private FlatFileItemWriter<AttributeReportGenerationDTO> flatFileItemWriter;

    @Override
    public void onSkipInProcess(AttributeReportGenerationDTO item, Throwable t) {

        try {
            LOGGER.error("Record not processed for attribute value with ID : " + item.getAttributeValueId());
            List<AttributeReportGenerationDTO> list = new ArrayList<AttributeReportGenerationDTO>();
            list.add(item);
            flatFileItemWriter.write(list);
        } catch (Exception e) {
            LOGGER.error("Unable to write to the error output file", e);
        }
    }

    /**
     * @param flatFileItemWriter
     *            the flatFileItemWriter to set
     */
    public void setFlatFileItemWriter(FlatFileItemWriter<AttributeReportGenerationDTO> flatFileItemWriter) {

        this.flatFileItemWriter = flatFileItemWriter;
    }

}

and my job configuration XML looks like this:

<bean id="skipListener" class="something.DTOProcessorListener" scope="step">
        <property name="flatFileItemWriter">
            <bean id="errorItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
                <property name="resource" value="file:#{jobParameters['error.filename']}" />
                <property name="appendAllowed" value="true" />
                <property name="lineAggregator">
                    <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                        <property name="fieldExtractor">
                            <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                                <property name="names"
                                    value="productTypeId, productTypeName, productId, productName, skuId, skuName, attributeValueId, attributeName, attributeValue, attributeType, nonEditableValueCheckSum, editableValueCheckSum" />
                            </bean>
                        </property>
                    </bean>
                </property>
                <property name="headerCallback">
                    <bean class="something.CsvHeaderImplementation">
                        <property name="headerString"
                            value="Product Type ID,Product Type,Product ID,Product Name,Sku ID,Sku Name,Attribute Value ID,Attribute Name,Attribute Value,Attribute Type,Check Sum 1,Check Sum 2" />
                    </bean>
                </property>
            </bean>
        </property>
    </bean>

I get the error

org.springframework.batch.item.WriterNotOpenException: Writer must be open before it can be written to

I am unable to set a stream entry in the job config as the bean for FlatFileItemWriter is internally specified (for the listener). If I create abean outside of the listener and refer to it, its returning a proxy instance of the FlatFileItemWriterClass.

Has anyone successfully wired up a writer to a flat file in the listener?

Thanks for the help

Michael Pralow Michael Pralow · Accepted Answer · 2011-09-30T18:58:00

Well why don't you use the writer as a normal bean ? You could register it as a stream and to get around the step proxy you could use the PropertPlaceholderConfigurer

i created a working example under my github repo, but i think spring batch could need an improvement here, it should be easier to implement error-item logging

Implementing SkipListener to write invalid records to a flat file

1 Answers