1
votes

I am going to convert ETL jobs (100's) to Mule ESB using DataWeave and other mule components. I am thinking on is there any preferred design on how to put these into mule application.

One or Two mule applications with flows for each job VS Individual mule application for each job

I am thinking about how would it make difference to memory consumption, contexts, threading etc.

Any thoughts?

Thanks.

3

3 Answers

2
votes

I think that having different applications or one app, it does not really make a difference in terms of performance. I think you should split taking in account this 2 factors

  • If jobs needs to share a lot of common logic and/or data than put it in the same app
  • Even if they don't need to share common logic but they serve the same business need than keep it in the same app

In addition to this consider also if some of this transformations could become real REST API where you can request data on demand and that can be potentially reused, if yes than make it so.

This are my 2 cents. Hope it helps.

1
votes

Both the options are possible - one, using one flow in an application and deployed independently and second, bundling a number of flows, multiple applications into a deployable.

The trade offs in the first case are - relatively more memory consumption, as each application starts in its own thread, more applications to manage, fine-grained control over managing these applications as they deployed independently.

In the second case, the memory consumption may be lower, it will be easy to manage fewer applications, but to make one change will require stopping other working ETL jobs as well.

Probably the best way would be to combine related ETL jobs into one application.

1
votes

I think keeping them separate makes it more scalable and manageable in long run with keeping in mind specific situations as @Mauro Rocco also mentioned

  1. Keeping together flows for same business need in same app
  2. I'd say put common flows and logic in separate commons app that can be referred by multiple apps.

With this approach, if an applications fails for some reason, other apps won't be impacted with ease of maintainability and testing and don't think memory consumption will be any different in any of approaches