I want to traverse and flatten a big JSON file that has following structure showing a product hierarchie (think of it as navigation in an online shop):
productGroups: [
{
"key": "child 1"
...
"childrenProductGroups": [
{
"key": "child 1.1",
...,
"childrenProductGroups": []
},
{
"key": "child 1.2"
...
"childrenProductGroups": [
{
"key": "child 1.2.1",
...,
"childrenProductGroups": [
{
"key": "child 1.2.1.1",
...,
childrenProductGroups": [
...
]
}
]
},
{
"key": "child 1.2.2",
...,
"childrenProductGroups": []
}
]
},
{
"key": "child 1.3",
...,
"childrenProductGroups": [
...
]
}
]
},
{
"key": "child 2",
...,
"childrenProductGroups": [
...
]
},
{
"key": "child 3",
...,
"childrenProductGroups": [
...
]
}
]
And I want to flatten them in a format like this:
{
"hierarchieSet": [
{
"Nodeid": "00000001", # Number in this json
"Nodename": "child 1",
"Tlevel": "01", # First child of product group
"Parentid": "00000000", # Parent is null
"Childid": "00000002", # Child node number
"Nextid": "00000008" # Node number on the same level (child 2)
},
{
"Nodeid": "00000002",
"Nodename": "child 1.1",
"Tlevel": "02",
"Parentid": "00000001",
"Childid": "00000003",
"Nextid": "00000003"
},
{
"Nodeid": "00000003",
"Nodename": "child 1.2",
"Tlevel": "02",
"Parentid": "00000002",
"Childid": "00000005",
"Nextid": "00000007"
},
{
"Nodeid": "00000004",
"Nodename": "child 1.2.1",
"Tlevel": "03",
"Parentid": "00000003",
"Childid": "0000005",
"Nextid": "00000006"
}
,
{
"Nodeid": "00000005",
"Nodename": "child 1.2.1.1",
"Tlevel": "04",
"Parentid": "00000004",
"Childid": "0000000", #No more children
"Nextid": "00000000"
},
{
"Nodeid": "00000006",
"Nodename": "child 1.2.2",
"Tlevel": "03",
"Parentid": "00000003",
"Childid": "0000000",
"Nextid": "00000000"
},
{
"Nodeid": "00000007",
"Nodename": "child 1.3",
"Tlevel": "02",
"Parentid": "00000001",
"Childid": "0000000",
"Nextid": "00000000"
},
{
"Nodeid": "00000008",
"Nodename": "child 2",
"Tlevel": "01",
"Parentid": "00000000",
"Childid": "0000009", # 00000009 not shown
"Nextid": "00000014" #
},
...
{
"Nodeid": "000000014",
"Nodename": "child 3",
"Tlevel": "01",
"Parentid": "00000000",
"Childid": "00000015",
"Nextid": "00000000" # 00000010 does not exist
}
]
}
Thus I have identified some main concerns:
- Recursion of the tree structure
- Transforming the elements
- Flattening the structure
- Keeping track of parents, siblings and children
- Keeping track of recursion level
- Formatting numbers
I tried to solve this issue by 2 different approaches:
- Use DataWeave to transform all elements
- Use Java to traverse the structure
As I'm fairly new to functional programming I put more focus on the Java implementation but ran into a number of issues.
Java approach
Read json > Init Tree var and assign the Java instance > for each element in top-level array invoke traverse(data, level)
in Tree.java
.
Tree.java:
import org.json.JSONObject;
public class Tree {
private int id = 0;
private List<Node> nodes = new ArrayList<Node>();
public Tree() {
nodes.add(new Node("01", "00000001", "HOME", "01", "00000000", "00000002", "00000000"));
}
public void traverse(String data, int level) {
System.out.println(data);
// TODO parse json
}
private void visit(JSONObject parent, JSONObject node, int level) {
id++;
nodes.add(new Node("01", String.valueOf(id), node.getString("key"), String.valueOf(level), "", "", ""));
}
public List<Node> getNodes() {
return nodes;
}
private static class Node {
private String zshop, nodename, parentid, childid, nextid, nodeid, tlevel;
public Node(String zshop, String nodeid, String nodename, String tlevel, String parentid, String childid, String nextid) {
this.zshop = zshop;
this.nodeid = nodeid;
this.nodename = nodename;
this.tlevel = tlevel;
this.parentid = parentid;
this.childid = childid;
this.nextid = nextid;
}
}
}
When calling the invoke action I use this payload:
%dw 2.0
output application/java
---
{
data: vars.rootMessage.payload as String,
level: 1
}
But this yields following error:
"Cannot coerce Object { encoding: UTF-8, mediaType: text/json; charset=UTF-8, mimeType: text/json, raw: org.mule.weave.v2.el.SeekableCursorStream@50ecee52 } (org.mule.weave.v2.el.MuleTypedValue@511ba9cc) to String
5| data: vars.rootMessage.payload as String, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Trace: at main (line: 5, column: 7)" evaluating expression: "%dw 2.0 output application/java --- { data: vars.rootMessage.payload as String, level: 1 }".
I tried a number of things:
- Cast it to a
ProductGroup
object I wrote in Java - Try to cast the object retrieved to
org.json.JSONObject
- Try to buffer and read
vars.rootMessage.payload (Binary)
But I wasn't able to solve it with any of these issues.
DataWeave approach My .dw Script
%dw 2.0
fun append
(item: Object, acc: Object = {
}) = acc ++ item
fun mapper(item: Object) =
{
Zshop: "01",
Nodeid: "00000000",
Nodename: item.key as Number as String {format: ""},
Tlevel: "02",
Parentid: "00000000",
Childid: "00000000",
Nextid: "00000000"
}
fun traverse(a: Array, level: Number) =
a map $ flatMap(value, index) -> value
output application/json
---
{
test: payload.productGroups reduce (item, acc) -> append(mapper(item), acc)
}
Where I tried to solve some of the problems. mapper(item)
should create json objects that I can append to the final output with appender(item, acc)
. Recursion has been sketched, but is not my main concern yet.
This yields this result:
(original payload),
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "800",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "110",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "720",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
"Zshop": "01",
"Nodeid": "00000000",
"Nodename": "710",
"Tlevel": "02",
"Parentid": "00000000",
"Childid": "00000000",
"Nextid": "00000000",
...
Where I wonder why I'm getting a flat result without any object structure.
My questions:
- Java: Why can't I cast the String or how is it done properly
- DataWeave: Is there an easy solution I don't see?
- Why is it a flat result and not an object?
- Are the usages of the
reduce
andflatMap
functions correct for this purpose?
Any help and / or feedback is welcome.