MuleSoft - query database, transform and map to XML, write to file

Question

I'm new to using MuleSoft. I've downloaded Anypoint Studio and am trying to create a proof of concept. I think I'm close, but not quite there. Basically here is the flow of what I want to occur:

Poll an oracle database to pull back x amount of records
Stream these records into a DataMapper, and have each record be placed in it's appropriate XML tag based off some xsd I provide
Write that XML output to some file on my local drive

What I would like is for a single record to be written to the xsd structure once every second, and generate a new file for that. So if there were 10 records in the DB, there would be 10 XML files generated for each time the database query is polled.

Here is an example of what I'm trying to accomplish based on the Configuration XML:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:batch="http://www.mulesoft.org/schema/mule/batch" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
    xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/batch http://www.mulesoft.org/schema/mule/batch/current/mule-batch.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd">
    <db:oracle-config name="#######" host="#######" port="####" instance="#######" user="#####" password="######" doc:name="Oracle Configuration"/>
    <file:connector name="XML_File" outputPattern="#[payload.planner_id]-#[server.dateTime]-.xml" autoDelete="true" streaming="true" validateConnections="true" doc:name="File" outputAppend="true" recursive="true" writeToDirectory="C:\Users\gm840d\Documents\Projects\IMM\AnypointStudio\FileOutput"/>
    <data-mapper:config name="List_Map__To_XML" transformationGraphPath="list_map__to_xml.grf" doc:name="List_Map__To_XML"/>
    <data-mapper:config name="List_Map__To_XML_1" transformationGraphPath="list_map__to_xml_1.grf" doc:name="List_Map__To_XML_1"/>
    <batch:job name="project_testxmlBatch" >
        <batch:input>
            <poll doc:name="Poll">
                <fixed-frequency-scheduler frequency="10" timeUnit="SECONDS"/>
                <processor-chain doc:name="Processor Chain">
                    <db:select config-ref="Ora_IMCT" doc:name="Database">
                        <db:parameterized-query><![CDATA[select planner, name, work_center
from schema.mytable]]></db:parameterized-query>
                    </db:select>
                </processor-chain>
            </poll>
        </batch:input>
        <batch:process-records>
            <batch:step name="Batch_Step" accept-policy="ALL">
                <batch:commit  doc:name="Batch Commit" size="306">
                    <data-mapper:transform config-ref="List_Map__To_XML_1" doc:name="List&lt;Map&gt; To XML"/>
                    <file:outbound-endpoint path="C:\Users\gm840d\Documents\Projects\IMM\AnypointStudio\FileOutput"  responseTimeout="10000" encoding="UTF-8" mimeType="text/xml" doc:name="File" connector-ref="XML_File"/>
                </batch:commit>
            </batch:step>
        </batch:process-records>
        <batch:on-complete>
            <logger message="Total Records exported: #[message.payload.getLoadedRecords()], Processing time: #[message.payload.getElapsedTimeInMillis()]app" level="WARN" doc:name="Logger"/>
        </batch:on-complete>
    </batch:job>
</mule>

It seems as though when I run the mule application, it works up to the point where it pulls in all the necessary records, but it stops after that. No files are generated or anything. Here is a snippet of the console output whenever the "poll" runs every few seconds.

*******************************************************************************************************
*            - - + APPLICATION + - -            *       - - + DOMAIN + - -       * - - + STATUS + - - *
*******************************************************************************************************
* project_testxml                               * default                        * DEPLOYED           *
*******************************************************************************************************

INFO  2015-02-27 10:41:23,124 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Created instance 3a62ec2d-beb0-11e4-adb3-ecf4bb3a71c5 for batch job project_testxmlBatch
INFO  2015-02-27 10:41:23,124 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Starting input phase
INFO  2015-02-27 10:41:23,124 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Input phase completed
INFO  2015-02-27 10:41:23,144 [pool-14-thread-1] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Starting loading phase for instance '3a62ec2d-beb0-11e4-adb3-ecf4bb3a71c5' of job 'project_testxmlBatch'
INFO  2015-02-27 10:41:23,236 [pool-14-thread-1] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Finished loading phase for instance 3a62ec2d-beb0-11e4-adb3-ecf4bb3a71c5 of job project_testxmlBatch. 306 records were loaded
INFO  2015-02-27 10:41:23,256 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Started execution of instance '3a62ec2d-beb0-11e4-adb3-ecf4bb3a71c5' of job 'project_testxmlBatch'
INFO  2015-02-27 10:41:32,256 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Created instance 3fd5e4a4-beb0-11e4-adb3-ecf4bb3a71c5 for batch job project_testxmlBatch
INFO  2015-02-27 10:41:32,256 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Starting input phase
INFO  2015-02-27 10:41:32,256 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Input phase completed
INFO  2015-02-27 10:41:32,256 [pool-14-thread-1] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Starting loading phase for instance '3fd5e4a4-beb0-11e4-adb3-ecf4bb3a71c5' of job 'project_testxmlBatch'
INFO  2015-02-27 10:41:32,327 [pool-14-thread-1] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Finished loading phase for instance 3fd5e4a4-beb0-11e4-adb3-ecf4bb3a71c5 of job project_testxmlBatch. 306 records were loaded
INFO  2015-02-27 10:41:32,327 [pool-14-thread-1] com.mulesoft.module.batch.engine.DefaultBatchEngine: Started execution of instance '3fd5e4a4-beb0-11e4-adb3-ecf4bb3a71c5' of job 'project_testxmlBatch'

Any help would be greatly appreciated. Been looking through Google forever to find an example like this and just not getting anywhere.

Mohan Mohan · Accepted Answer · 2015-02-28T09:35:34

in batch-step of batch process records, add collection splitter before the datmapper and implement the logic you specified for xsd definition for each record format and write it to file as you specified.

MuleSoft - query database, transform and map to XML, write to file

2 Answers