Play framework (Scala) - Getting subset of json that contains arrays

Question

I have many very large json-objects that I return from Play Framework with Scala.

In most cases the user doesn't need all the data in the objects, only a few fields. So I want to pass in the paths I need (as query parameters), and return a subset of the json object.

I have looked at using JSON Transformers for this task.

Filter code

def filterByPaths(paths: List[JsPath], inputObject: JsObject) : JsObject = {
  paths
    .map(_.json.pick)
    .map(inputObject.transform)
    .filter(_.isSuccess)
    .map { case JsSuccess(value, path) => (value, path) }
    .foldLeft(Json.obj()) { (obj, jsValueAndPath) =>
      val(jsValue, path) = jsValueAndPath
      val transformer = __.json.update(path.json.put(jsValue))
      obj.transform(transformer).get
    }
}

Usage:

val input = Json.obj(
  "field1" -> Json.obj(
    "field2" -> "right result"
  ),
  "field4" -> Json.obj(
    "field5" -> "not included"
  ),
)

val result = filterByPaths(List(JsPath \ "field1" \ "field2"), input)
// {"field1":{"field2":"right result"}}

Problem

This code works fine for JsObjects. But I can't make it work if there are JsArrays in the strucure. I had hoped that my JsPath could contain an index to look up the field, but that's not the case. (Don't know why I assumed that, maybe my head was too far in the JavaScript-world)

So this would fail to return the first entry in the Array:

val input: JsObject = Json.parse("""
  {
    "arr1" : [{
      "field1" : "value1"
    }]
  }
  """).as[JsObject]

val result = filterByPaths(List(JsPath \ "arr1" \ "0"), input)
// {}

Question

My question is: How can I return a subset of a json structure that contains arrays?

Alternative solution

I have the data as a case class first, and I serialize it to Json, and then run filterByPaths on it. Having a Reader that only creates the json I need in the first place might be a better solution, but creating a Reader on the fly, with configuration from queryparams seamed a more difficult task, then just stripping down the json afterwards.

Andriy Kuba Andriy Kuba · Accepted Answer · 2019-01-17T12:07:09

The example of the returning array element:

val input: JsValue = Json.parse("""
  {
    "arr1" : [{
      "field1" : "value1"
    }]
  }
  """)

val firstElement = (input \ "arr1" \ 0).get
val firstElementAnotherWay = input("arr1")(0)

More about this in the Play Framework documentation: https://www.playframework.com/documentation/2.6.x/ScalaJson

Update

It looks like you got the old issue RuntimeException: expected KeyPathNode. JsPath.json.put, JsPath.json.update can't past an object to a nesting array.

https://github.com/playframework/playframework/issues/943

https://github.com/playframework/play-json/issues/82

What you can do:

Use the JSZipper: https://github.com/mandubian/play-json-zipper
Create a script to update arrays "manually"
If you can afford it, strip array in a resulting object

Example of stripping array (point 3):

def filterByPaths(paths: List[JsPath], inputObject: JsObject) : JsObject = {
  paths
    .map(_.json.pick)
    .map(inputObject.transform)
    .filter(_.isSuccess)
    .map { case JsSuccess(value, path) => (value, path)}
    .foldLeft(Json.obj()) { (obj, jsValueAndPath) =>
      val (jsValue, path) = jsValueAndPath
      val arrayStrippedPath = JsPath(path.path.filter(n => !(n.toJsonString matches """\[\d+\]""")))
      val transformer = __.json.update(arrayStrippedPath.json.put(jsValue))
      obj.transform(transformer).get
    }
}

val result = filterByPaths(List(JsPath \ "arr1" \ "0"), input)
// {"arr1":{"field1":"value1"}}

The example

Play framework (Scala) - Getting subset of json that contains arrays

2 Answers

Update