Deserializing an enum using a combination of #[serde(untagged)] and #[serde(with)]

Question

I'm trying to use an actix-web server as a gateway to a small stack to guarantee a strict data format inside of the stack while allowing some freedoms for the user.

To do that, I want to deserialize a JSON string to the struct, then validate it, serialize it again and publish it on a message broker. The main part of the data is an array of arrays that contain integers, floats and datetimes. I'm using serde for deserialization and chrono to deal with datetimes.

I tried using a struct combined with an enum to allow the different types:

#[derive(Serialize, Deserialize)]
pub struct Data {
    pub column_names: Option<Vec<String>>,
    pub values: Vec<Vec<ValueType>>,
}

#[derive(Serialize, Deserialize)]
#[serde(untagged)]
pub enum ValueType {
    I32(i32),
    F64(f64),
    #[serde(with = "datetime_handler")]
    Dt(DateTime<Utc>),
}

Since chrono::DateTime<T> does not implement Serialize, I added a custom module for that similar to how it is described in the serde docs.

mod datetime_handler {
    use chrono::{DateTime, TimeZone, Utc};
    use serde::{self, Deserialize, Deserializer, Serializer};

    pub fn serialize<S>(dt: &DateTime<Utc>, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        let s = dt.to_rfc3339();
        serializer.serialize_str(&s)
    }

    pub fn deserialize<'de, D>(deserializer: D) -> Result<DateTime<Utc>, D::Error>
    where
        D: Deserializer<'de>,
    {
        println!("Checkpoint 1");
        let s = String::deserialize(deserializer)?;
        println!("{}", s);
        println!("Checkpoint 2");
        let err1 = match DateTime::parse_from_rfc3339(&s) {
            Ok(dt) => return Ok(dt.with_timezone(&Utc)),
            Err(e) => Err(e),
        };
        println!("Checkpoint 3");

        const FORMAT1: &'static str = "%Y-%m-%d %H:%M:%S";
        match Utc.datetime_from_str(&s, FORMAT1) {
            Ok(dt) => return Ok(dt.with_timezone(&Utc)),
            Err(e) => println!("{}", e), // return first error not second if both fail
        };
        println!("Checkpoint 4");
        
        return err1.map_err(serde::de::Error::custom);
    }
}

This tries 2 different time formats one after the other and works for DateTime strings.

The Problem

It seems like the combination of `#[derive(Serialize, Deserialize)]`, `#[serde(untagged)]` and `#[serde(with)]` does something unexpected. `serde:from_str(...)` tries to deserialize every entry in the array with my custom `deserialize` function. I would expect it to either try to deserialize into `ValueType::I32` first, succeed and continue with the next entry, as [the docs](https://serde.rs/enum-representations.html) say:

Serde will try to match the data against each variant in order and the first one that deserializes successfully is the one returned.

What happens is that the custom deserializeis applied to e.g. "0" fails and the deserialization stops.

What's going on? How do I solve it?

My ideas are that I either fail to deserialize in the wrong way or that I somehow "overwrite" the derived deserialize with my own.

I cannot understand your problem. It works for me in this playground. Maybe you could post the JSON you are trying to deserialize and not just the Rust types. play.rust-lang.org/… — jonasbb
@jonasbb it only happens if the they are strings representing numbers (which might already be the solution to the problem) like so play.rust-lang.org/… — Haifischbecken
Serde will not perform type transformations automatically. If it sees a JSON string it will map that to a Rust string. I like to use the DisplayFromStr type from serde_with for such transformation tasks. docs.rs/serde_with/1.9.4/serde_with/struct.DisplayFromStr.html — jonasbb
Looks great, I found a little crate that does what I want called serde-aux, but I'll have a look at this variant, fewer dependencies are always welcome. — Haifischbecken

Haifischbecken Haifischbecken · Accepted Answer · 2021-07-09T12:41:42

@jonasbb helped me realize the code works when using [0,16.9,"2020-12-23 00:23:14"] but it does not when trying to deserialize ["0","16.9","2020-12-23 00:23:14"]. Serde does not serialize numbers from strings by default, the attempts for I32 and F64 just fail silently. This is discussed in this serde-issue and can be solved using the inofficial serde-aux crate.

Deserializing an enum using a combination of #[serde(untagged)] and #[serde(with)]

The Problem

2 Answers