XQuery file returning invalid entity reference using special characters

Question

I have the following query in a MarkLogic XQuery file, and I am seeing the following error message returned

XDMP-ENTITYREF: (err:XPST0003) Invalid entity reference " " . See the MarkLogic server error log for further detail.

The following is the code I am using in the XQuery file.

xquery version "1.0-ml";

declare variable $query := 

  cts:or-query
  ((
    cts:element-word-query(xs:QName("lines"),"l&amp;l"),
    cts:element-word-query(xs:QName("lines"),"pool &amp; cue"),
    cts:element-word-query(xs:QName("lines"),"look")
  ));

declare function local:do-query(){
  element xml {
    for $i in cts:uris( (), (), $query)
    let $item := doc($i)

    return
      element item {
        element title { $item/title/string() }
    }
  }
};

local:do-query()

Obviously the 2x tags i am looking for are l&l and pool & cue. I have also looked into the repair-full suggestion in another question posted, but couldn't figure out how that fits into this query. If I removed the ones with special characters, it works as expected.

Any ideas?

This code runs just fine in QConsole, so perhaps there is a different issue. How are you invoking the code, and are you sure you are invoking above code and not something else? Could it be a data issue instead? — grtjn
Yes the code runs fine in qconsole agreed, but when uploading said code into a .xqy file and trying to run it in a browser, it's there where the error message is returned. Expected behaviour is to see the xml results as observed via the console. — Key
How did you upload the .xqy file, and did you check with for instance QConsole Explore feature on your modules database whether the .xqy file once loaded into the database looks correct? — grtjn
I'm using 'xdmp:document-insert' to insert the file to the modules database. Great point regarding how it shows up in the .xqy file. Both element word queries are showing up as 'l & l' and 'pool & cue', which isn't the expected behaviour either. — Key
Yes, it is less complicated to load .xqy files from disk when inserting or uploading them. There are tools that can help with deployment, and you can use rest apis for that too. — grtjn

grtjn grtjn · Accepted Answer · 2017-04-10T17:38:04

Based on the additional info in the comments to the question, this is not an issue with the execution of the code, but rather with deployment of the code.

This happens often if you insert code using QConsole, or some other ways in which you evaluate XQuery code. The & get interpreted, and translated to the & character it represents. If you then write that into a .xqy file into some Modules database, it does not get escaped back into & again, since XQuery files are stored as plain text in MarkLogic, and & doesn't get escaped in plain text.

A better way to deploy code is by uploading or inserting from disk. That way characters like &, >, and { inside XML won't get interpreted, but preserved and inserted as is. There are tools like ml-gradle and Roxy that make deploying MarkLogic code very easy. Consider using these. Alternatively you could also look into using Curl against the Management REST api.

If you want to use QConsole after all, escape characters like & twice. E.g. & becomes &amp;, and < becomes &lt;.

HTH!

XQuery file returning invalid entity reference using special characters

1 Answers