how to read data from multiple tables in db using spring batch

Question

I tried reading data from one table and writing to other table using spring batch but now my requirement is to read data from mutiple tables and write to a file, so we can achieve this by defining mutiple jobs but I want to do it using single job means single reader and single writer and single processor.

Please provide me some references for this scenario.

You can use "CompositeWriter" of Spring Batch, see here stackoverflow.com/questions/15805868/… — Shailendra
ok but what we should use for reader to read from multiple tables — User111
What reader are you using? I've used a MyBatis reader binded with a sql, which reads data from multiple tables. — songyuanyao
If the tables are in the same database, why can't you construct a SQL query that returns all the data required? — Michael Minella

NIVESH SENGAR NIVESH SENGAR · Accepted Answer · 2015-06-12T10:22:46

Not possible by the classes provided by the spring batch but you can make a way our of it.
Just before the chunk processing add one step, make a custom tasklet where you will assign different sql and different output file and make them run in loop as long as there are sqls to execute.
It might sound difficult but I have worked on same situation, Here is some idea how you can do it -

   <flow id="databaseReadWriteJob">
            <step id="step1_setReaderWriter">
                <tasklet ref="setReaderWriter" />
                <next on="FAILED" to="" />
                <next on="*" to="dataExtractionStep" />
            </step>
            <step id="dataExtractionStep">
                <tasklet>
                    <chunk reader="dbReader" writer="flatFileWriter" commit-interval="${commit-interval}" />
</tasklet>
                <next on="FAILED" to="" />
                <next on="*" to="step3_removeProcessedSql" />
            </step>
            <step id="step3_removeProcessedSql">
                <tasklet ref="removeProcessedSql" />
                <next on="NOOP" to="step1_setReaderWriter" />
                <next on="*" to="step4_validateNumOfSuccessfulSteps" />
            </step>
    </flow>

and here is the bean for setReaderWriter

<beans:bean id="setReaderWriter" class="SetReaderWriter">
        <beans:property name="reader" ref="dbReader" />
        <beans:property name="flatFileWriter" ref="flatFileWriter" />
        <beans:property name="fileSqlMap" ref="jobSqlFileMap" />
        <beans:property name="fileNameBuilder" ref="localFileNameBuilder" />
        <beans:property name="sourceFolder" value="${dataDir}" />
        <beans:property name="dateDiff" value="${dateDiff}" />

Anything you need to add dynamically in Reader or Writer. Above sqlMap is the map of sql as key and Output file as value of that.
I hope it could help.

how to read data from multiple tables in db using spring batch

1 Answers