Orbeon: Using a filesystem for storing Form Builder forms

Question

We have started using Orbeon Form Builder to create forms, however we would like to be able to use source control (Git) to manage the forms we create. As such, we would like to store the forms in the file system instead of in a database (as a side note, data from the forms is being handled separately).

I have seen that the/a way to do this would be to create a (custom) persistence layer for it (http://doc.orbeon.com/configuration/properties/persistence.html http://doc.orbeon.com/form-runner/api/persistence/index.html).

Is there a documented/available way to easily do this without having to reinvent the wheel (Orbeon 4.2 Form Builder makes /crud/orbeon/builder/form/form.xhtml request to the custom persistence layer)?

Answer 1

One way to achieve this would be to leverage the resource persistence layer. Currently, the resource persistence layer only supports read operations, and it would need to be improved to also support write operations. With that, you would be able to create, edit, and publish forms with Form Builder. This would write files on disk, which you could then separately add/commit/push with Git.

(The persistence layer could also do those operations on Git for you, but I would imagine that most Git users would like more control there than would be provided by a fully automatic system.)

Answer 2

For completion, here is what we have implemented.

We are using a Git repository to store the changes we make to the Orbeon installation (on a new Orbeon installation, we initialise the folder as a git repository, add our repository as a remote repository, then pull in the repository).

For development instances, we are using the default eXist database, so developers have to manually copy the forms into form builder when they first start, or when someone else makes changes to the form.

When developers have a change ready to commit to the repository, they can pull the form (both the draft version and the published version) using a Node.js script (below), which, using the CRUD API, grabs the XML and stores it in the correct places for the resource persistence layer.

In non-development instances, the resource persistence layer is used instead of eXist, so Orbeon grabs the forms from the file system (WEB-INF/resource/forms/<APP_NAME>/<FORM_NAME>/form/form.xhtml). We are not storing any data using Orbeon (this is all handled by other services).

This works well enough for us at the moment - if two people have worked on the same form, merges can get quite daunting as form xml is not the... simplest. The differences in development/non-development environments and stored in properties-local-dev.xml and properties-local-prod.xml (which is selected by change the run mode (in WEB-INF/web.xml)

Node.js getForm.js script

#!/usr/bin/env node

var get = require('get');
var mkdirp = require('mkdirp');
var fs = require('fs');
var path = require('path');

if (process.argv.length !== 6) {
  console.error("usage: getForm.sh <url> <application> <form> <editFormId>");
  console.error("Gets the latest version of a form from Orbeon");
  console.error("<editFormId> is the id of the latest version of the form in form ");
  console.error("  eg, the last part of the edit form url");
  console.error("  http://localhost/orbeon/fr/orbeon/builder/edit/<editFormId>");
  console.error("eg: getForm.sh http://127.0.0.1:8080/orbeon APP FORM 72e300fe7d8f2d2604f31b690760865a23dbeaed");
  return 1;
}


// Check for WEB-INF folder so we know we are running from the right folder
try {
  fs.accessSync('WEB-INF', fs.R_OK);
} catch (err) {
  console.error('Problem finding WEB-INF folder. Need to run from base of '
      + 'Orbeon folder', err.stack);
  return 2;
}

// Create folders if required
var publishedFolder = path.join('WEB-INF/resources/forms/', process.argv[3],
    process.argv[4], 'form');
var draftFolder = path.join('src/forms/', process.argv[3]);
var folders = [publishedFolder, draftFolder];
var f;

for (f in folders) {
  try {
    fs.accessSync(folders[f], fs.R_OK);
  } catch (err) {
    if (err.code === 'ENOENT') {
      // Try creating the folder
      try {
       mkdirp.sync(folders[f]);
      } catch (err) {
        console.error("Error creating folder", folders[f], err.stack);
        return 3;
      }
    } else {
      console.error("Error checking access permissions to folder", folders[f], err.stack);
      return 4;
    }
  }
}

try {
  // Add http to url if it doesn't already contain it
  if (!process.argv[2].match(/^https?:\/\//)) {
    process.argv[2] = 'http://' + process.argv[2];
  }
  var publishedURL = process.argv[2] + '/fr/service/persistence/'
      + 'crud/' + process.argv[3] + '/' + process.argv[4] + '/form/form.xhtml';
  var publishedFile = path.join(publishedFolder, 'form.xhtml');

  console.log('Getting the latest published version of the form from '
      + publishedURL + ' and saving it to', publishedFile);

  var published = get(publishedURL);
  // Write to file

  published.toDisk(publishedFile, function(err, filename) {
    if (err) {
      console.error('Error storing the latest published version of the form',
          publishedURL, ' > ', publishedFile, err.stack);
    }
    return 5;
  });
} catch (err) {
  console.error('Error getting the latest published version of the form',
      publishedURL, ' > ', publishedFile, err.stack);
  return 6;
}

try {
  var draftURL = process.argv[2] + '/fr/service/persistence/crud/'
      + 'orbeon/builder/data/' + process.argv[5] + '/data.xml';
  console.log(draftURL);
  var draftFile = path.join(draftFolder,
      process.argv[4] + '.xhtml');

  console.log('Getting the latest draft version of the form from '
      + draftURL + ' and saving it to', draftFile);

  var draft = get(draftURL);
  // Write to file

  draft.toDisk(draftFile, function(err, filename) {
    if (err) {
      console.error('Error storing the latest draft version of the form',
          draftURL, ' > ', draftFile, err.stack);
    }
    return 7;
  });
} catch (err) {
  console.error('Error getting the latest draft version of the form',
      draftURL, ' > ', draftFile, err.stack);
  return 8;
}

To enable access to the CRUD API

<property
  as="xs:string"
  processor-name="oxf:page-flow"
  name="page-public-methods"
  value="GET HEAD POST PUT DELETE"/>

<property
  as="xs:string"
  processor-name="oxf:page-flow"
  name="service-public-methods"
  value="GET HEAD POST PUT DELETE"/>