MarkLogic : Design question on Query Options Vs Transform

Question

Our team creates a bunch of custom REST APIs (v1/resources/...) and expose them as enterprise services to other stakeholders, who do not need to know anything about MarkLogic. However, our team is responsible for creating, enhancing and maintaining the server-side scripting (we use JavaScript) within MarkLogic.

While creating custom REST APIs, our current design to meet the search requirements is to start with a Query Options, incorporate as many requirements in Query options, and for any requirements that could not be met by Query Options (for example, sorting within a document, complex XPath, merge with other documents etc.), code within the Java Script extension program (technically not a transform but conceptually similar to a transform).

With the limitations in Query options, increasingly, most of our logic is going into the Javascript extension programs and the query options seem to be just a maintenance overhead. Do we really need to maintain a query options file for every REST extension, while the transforms offer much powerful functionality? Can I get rid of Query options and just use server side Java Script code (conceptually, similar to transforms)? Initially, our thought was that Query options is configuration based and hence changing query options is not exactly a code change, however, based on our experience, we realized that changing query options also involves deployment, regression testing and all other activities. Hence I do not see any specific advantage of query options, in our case (creating custom REST APIs).

Design gurus, please suggest!

Answer 1

These questions are tough to answer without properly knowing the whole situation. You talk about custom REST endpoints, but are using REST extensions on the built-in REST api for that purpose. Are you using anything else of the built-in REST api? You use search options, but don't use them for /v1/search. Could you have done the same with /v1/search with those options, but an extra REST Transform on top? And you speak of a lot of data mangling happening at sub-document level. Have you considered doing part of that processing upfront, or organize your data differently so that handling at request-time becomes much simpler?

A lot of questions, and a lot of possibilities. It is hard to give one straight answer on such an open question. I hope I gave some food for thought nonetheless.

HTH!