JSONObject initialized with a HashMap containing an ArrayList is not initialized correct on some Android 4 versions - how to fix it?

Question

Context

An Android application using Couchbase Lite (http://developer.couchbase.com/documentation/mobile/1.3/develop/references/couchbase-lite/couchbase-lite/index.html) to load json document(s). For example an questionnaire containing a list of questions:

{
    "questions": [
        {
            "answer": 1,
            "attributeName": "baiOne",
            "text": "Some text",
            "timeOfAnswer": {
                "timezoneOffset": -60,
                "utcTime": 1458118338993
            }
        },
        ...
    ],
    "endDate": 1458604800000,
    "startDate": 1458086400000,
    "title": "BAI"
}

This is correctly loaded with

Document doc = database.getDocument(docId);
Map<String, Object> props =
            new HashMap<String, Object>(doc.getProperties());

I can check it with some asserts directly:

assert(props.containsKey("questions") != true)
assert(props.get("questions").getClass().getName() != ArrayList.class.getName());

Problem

Ok perfect lets jump to the trouble maker:

JSONObject result = new JSONObject(props);

On Android 4.4.2 (api-level 19) and up it looks correct with:

• The questions attribute (ArrayList) in the result JSONObject is generated as an JSONArray
• result.toString() is logged: {"questions":[{"answer":1,"attributeName":"baiOne"}]}

But for Android 4.3 (api-level 18) and down to 4.1.2 (api-level 16) it does not look correct:

• The questions attribute (ArrayList) in the result JSONObject is generated as an JSONObject
  • result.toString() is logged: {"questions":"[{attributeName=baiOne, answer=1}]"} where the entire questions block is turned into one big string.

Not convinced ? Perfect lets look at a simple regression test that I have used for an Android instrumentation test:

@Test
public void testRegression_hashmap_to_json_object()
        throws JSONException, CouchbaseLiteException, IOException {

    Log.v(TAG, "Create data that mimic a json document loaded from coucbase lite:");
    Map<String, Object> question = new HashMap<String, Object>();
    question.put("answer", 1);
    question.put("attributeName", "baiOne");
    List<Map<String, Object>> questions = new ArrayList<Map<String, Object>>();
    questions.add(question);

    Map<String, Object> properties = new HashMap<String, Object>();
    properties.put("questions", questions);

    Log.v(TAG, "Create JSONObject from properties hashmap:");
    JSONObject t = new JSONObject(properties);

    // Do stuff with t ...

    Log.v(TAG, "t.toString(): " + t.toString());
    // prints on android 4.1.2 (16) - 4.3 (18) prints incorrectly:
    // t.toString(): {"questions":"[{attributeName=baiOne, answer=1}]"}
    // android 4.4.2 (19) and up prints correctly:
    // t.toString(): {"questions":[{"answer":1,"attributeName":"baiOne"}]}

    JSONArray a = t.getJSONArray("questions");
    assertTrue(a != null);
    assertTrue(a.length() == 1);

    // Save doc back to couchbase lite
}

Possible related stackoverflow (which do not solve the problem where an ArrayList is not turned into a JSONArray):

• Android 4 JSON generation bug: Can I use a newer version of the org.JSON library than the bundled one?

Finally any pointers for a simple solution?

Edit 1

Try out with a naive modified version of Orion Edwards workaround with a custom stringifier. It handles conversion of ArrayList to JSONArray (warning: my java type casting skills are rusty):

private JSONObject stringify(Map<String,Object> map) {
    Map<String,Object> fixed = new HashMap<String,Object>();
    for (String key : map.keySet()){
        Object value = map.get(key);
        if (value instanceof Map) {
            value = stringify((Map<String,Object>) value);
        }
        if (value instanceof List) {
            value = stringify(value);
        }
        fixed.put(key,value);
    }
    return new JSONObject(fixed);
}

private JSONArray stringify(Object value) {
    JSONArray a = new JSONArray();
    if (!((List) value).isEmpty() && ((List) value).get(0) instanceof Map) {
        for (Map<String, Object> m: (List<Map<String, Object>>) value) {
            a.put(stringify(m));
        }
    }
    else {
        for (Object m: (List<Object>) value) {
            a.put(m);
        }
    }
    return a;
}

Answer 1

Based on the feedback from @borrrden and the couchbase lite api / tutorials (for example http://developer.couchbase.com/documentation/mobile/1.2/develop/training/build-first-android-app/do-crud/index.html) the solution is to purely use new HashMap<String, Object>() and not to use JSONObject at all.