Why Kotlin data classes can have nulls in non-nullable fields with Gson?

Question

In Kotlin you can create a data class:

data class CountriesResponse(
    val count: Int,
    val countries: List<Country>,
    val error: String)

Then you can use it to parse a JSON, for instance, "{n: 10}". In this case you will have an object val countries: CountriesResponse, received from Retrofit, Fuel or Gson, that contains these values: count = 0, countries = null, error = null.

In Kotlin + Gson - How to get an emptyList when null for data class you can see another example.

When you later try to use countries, you will get an exception here: val size = countries.countries.size: "kotlin.TypeCastException: null cannot be cast to non-null type kotlin.Int". If you write a code and use ? when accessing these fields, Android Studio will highlight ?. and warn: Unnecessary safe call on a non-null receiver of type List<Country>.

So, should we use ? in data classes? Why can an application set null to non-nullable variables during runtime?

Answer 1

This happens because Gson uses an unsafe (as in java.misc.Unsafe) instance construction mechanism to create instances of classes, bypassing their constructors, and then sets their fields directly.

See this Q&A for some research: Gson Deserialization with Kotlin, Initializer block not called.

As a consequence, Gson ignores both the construction logic and the class state invariants, so it is not recommended to use it for complex classes which may be affected by this. It ignores the value checks in the setters as well.

Consider a Kotlin-aware serialization solution, such as Jackson (mentioned in the Q&A linked above) or kotlinx.serialization.

Answer 2

It's a design decision of many Json parsers, including Gson, to make it silently swallow absent fields. Objects are being set to null and primitives are being set to 0, completely masking API errors and causing cryptic errors further downstream.

For this reason, I recommend 2-stage parsing:

package com.example.transport
//this class is passed to Gson (or any other parser)
data class CountriesResponseTransport(
   val count: Int?,
   val countries: List<CountryTransport>?,
   val error: String?){

   fun toDomain() = CountriesResponse(
           count ?: throw MandatoryIsNullException("count"),
           countries?.map{it.toDomain()} ?: throw MandatoryIsNullException("countries"),
           error ?: throw MandatoryIsNullException("error")
       )
}

package com.example.domain
//this one is actually used in the app
data class CountriesResponse(
   val count: Int,
   val countries: Collection<Country>,
   val error: String)

Yes, it's twice as much work - but it pinpoints API errors immediately and gives you a place to handle those errors if you can't fix them, like:

   fun toDomain() = CountriesResponse(
           count ?: countries?.count ?: -1, //just to brag we can default to non-zero
           countries?.map{it.toDomain()} ?: ArrayList()
           error ?: MyApplication.INSTANCE.getDeafultErrorMessage()
       )

Yes, you can use a better parser, with more options - but you shouldn't. What you should do is abstract the parser away so you can use any. Because no matter how advanced and configurable parser you find today, eventually you'll need a feature that it doesn't support. That's why I treat Gson as the lowest common denominator.

There's an article that explains this concept used (and expanded) in a bigger context of repository pattern.