Pattern matching on provided types

Question

Firstly, obtain a schema and parse:

type desc = JsonProvider< """[{"name": "", "age": 1}]""", InferTypesFromValues=true >
let json = """[{"name": "Kitten", "age": 322}]"""
let typedJson = desc.Parse(json)

Now we can access typedJson.[0] .Age and .Name properties, however, I'd like to pattern match on them at compile-time to get an error if the schema is changed.

Since those properties are erased and we cannot obtain them at run-time:

let ``returns false``() = 
  typedJson.[0].GetType()
    .FindMembers(MemberTypes.All, BindingFlags.Public ||| BindingFlags.Instance, 
                 MemberFilter(fun _ _ -> true), null) 
  |> Array.exists (fun m -> m.ToString().Contains("Age"))

...I've made a runtime-check version using active patterns:

let (|Name|Age|) k = 
  let toID = NameUtils.uniqueGenerator NameUtils.nicePascalName
  let idk = toID k
  match idk with
  | _ when idk.Equals("Age") -> Age
  | _ when idk.Equals("Name") -> Name
  | ex_val -> failwith (sprintf "\"%s\" shouldn't even compile!" ex_val)

typedJson.[0].JsonValue.Properties()
|> Array.map (fun (k, v) -> 
     match k with
     | Age -> v.AsInteger().ToString() // ...
     | Name -> v.AsString()) // ... 
|> Array.iter (printfn "%A")

In theory, if FSharp.Data wasn't OS I wouldn't be able to implement toID. Generally, the whole approach seems wrong and redoing the work.

I know that discriminated unions can't be generated using type providers, but maybe there's a better way to do all this checking at compile-time?

Answer 1

As far as I know it cannot be possible to find out if "Json schema has changed" at compile-time using the given TP.

That's why:

JsonProvider<sample> is exactly what kicks in at compile-time providing a type for manipulating Json contents at run-time. This provided erased type has couple of run-time static methods common for any sample and type Root extending IJsonDocument with few instance properties including ones based on compile-time provided sample (in your case - properties Name and Age).There is exactly one very relaxed implicit Json "schema" behind JsonProvider-provided type, no another such entity to compare with for change at compile-time;

at run-time only provided type desc with its static methods and its Root type with correspondent instance methods are at your service for manipulating arbitrary Json contents. All this jazz is pretty much agnostic with regard to "Json schema", in your given case as long as run-time Json contents represent an array its elements may be pretty much any. For example,

type desc = JsonProvider<"""[{"name": "", "age": 1}]"""> // @ compile-time

let ``kinda "typed" json`` = desc.Parse("""[]""") // @ run-time
let ``another kinda "typed" json`` =
    desc.Parse("""[{"contents":"whatever", "text":"blah-blah-blah"},[{"extra":42}]]""")

both will be happily parsed at run-time as "typed Json" conforming to "schema" derived by TP from the given sample, although apparently Name and Age are missing and exceptions will be raised if accessed.

Json schema

In this case change of the referred schema may break compilation if provided accessors being used in the code are incompatible with the change. Such arrangement accompanied by run-time Json payload validator or validating parser may provide reliable enterprise-quality Json schema change management.

JsonProvider TP from Fsharp.Data lacks such Json schema handling abilities, so payload validations are to be done in run-time only.

Answer 2

Quoting your comments which explain a little better what you are trying to achieve:

Thank you! But what I'm trying to achieve is to get a compiler error if I add a new field e.g. Color to the json schema and then ignore it while later processing. In case of unions it would be instant FS0025.

and:

yes, I must process all fields, so I can't rely on _. I want it so when the schema changes, my F# program won't compile without adding necessary handling functionality(and not just ignoring new field or crashing at runtime).

The simplest solution for your purpose is to construct a "test" object.

The provided type comes with two constructors: one takes a JSonValue and parses it - effectively the same as JsonValue.Parse - while the other requires every field to be filled in.

That's the one that interests us.

We're also going to invoke it using named parameters, so that we'll be safe not only if fields are added or removed, but also if they are renamed or changed.

type desc = JsonProvider< """[{"name": "SomeName", "age": 1}]""", InferTypesFromValues=true >

let TestObjectPleaseIgnore = new desc.Root (name = "Kitten", age = 322)
// compiles

(Note that I changed the value of name in the sample to "SomeName", because "" was being inferred as a generic JsonValue.)

Now if more fields suddenly appear in the sample used by the type provider, the constructor will become incomplete and fail to compile.

type desc = JsonProvider< """[{"name": "SomeName", "age": 1, "color" : "Red"}]""", InferTypesFromValues=true >

let TestObjectPleaseIgnore = new desc.Root (name = "Kitten", age = 322)
// compilation error: The member or object constructor 'Root' taking 1 arguments are not accessible from this code location. All accessible versions of method 'Root' take 1 arguments.

Obviously, the error refers to the 1-argument constructor because that's the one it tried to fit, but you'll see that now the provided type has a 3-parameter constructor replacing the 2-parameter one.

Answer 3

If you use InferTypesFromValues=false, you get a strong type back:

type desc = JsonProvider< """[{"name": "", "age": 1}]""", InferTypesFromValues=false >

You can use the desc type to define active patterns over the properties you care about:

let (|Name|_|) target (candidate : desc.Root) =
    if candidate.Name = target then Some target else None

let (|Age|_|) target (candidate : desc.Root) =
    if candidate.Age = target then Some target else None

These active patterns can be used like this:

let json = """[{"name": "Kitten", "age": 322}]"""
let typedJson = desc.Parse(json)

match typedJson.[0] with
| Name "Kitten" n -> printfn "Name is %s" n
| Age 322m a -> printfn "Age is %M" a
| _ -> printfn "Nothing matched"

Given the typedJson value here, that match is going to print out "Name is Kitten".

Answer 4

<tldr>
Build some parser to handle your issues. http://www.quanttec.com/fparsec/
</tldr>

So...

You want something that can read something and do something with it. Without knowing apriori what any of those somethings is.

Good luck with that one.

You do not want type provider to do this for you. Type providers are made with the full purpose of being "at compile time this is what I saw, and thats what I will use".

With that said:

You want some other type of parser, where you are able to check the "schema" (some definition of what you know is going to come or you saw last time vs. what actually came). In effect some dynamic parser getting the data to some dynamic structure, with dynamic types.

And mind you, dynamic is not static. F# has static types and a lot is based on that. And type providers more so. Fighting that will make your head ache. It is of course possible, and it might even be possible by fighting the type providers to actually work with such an approach, but then again its not really a type provider nor its purpose.