Deserializing newline-delimited JSON from a socket using Serde

Question

I am trying to use serde for sending a JSON struct from a client to a server. A newline from the client to the server marks that the socket is done. My server looks like this

#[derive(Serialize, Deserialize, Debug)]
struct Point3D {
    x: u32,
    y: u32,
    z: u32,
}

fn handle_client(mut stream: TcpStream) -> Result<(), Error> {
    println!("Incoming connection from: {}", stream.peer_addr()?);
    let mut buffer = [0; 512];
    loop {
        let bytes_read = stream.read(&mut buffer)?;
        if bytes_read == 0 {
            return Ok(());
        }
        let buf_str: &str = str::from_utf8(&buffer).expect("Boom");
        let input: Point3D = serde_json::from_str(&buf_str)?;
        let result: String = (input.x.pow(2) + input.y.pow(2) + input.z.pow(2)).to_string();
        stream.write(result.as_bytes())?;
    }
}

fn main() {
    let args: Vec<_> = env::args().collect();
    if args.len() != 2 {
        eprintln!("Please provide --client or --server as argument");
        std::process::exit(1);
    }
    if args[1] == "--server" {
        let listener = TcpListener::bind("0.0.0.0:8888").expect("Could not bind");
        for stream in listener.incoming() {
            match stream {
                Err(e) => eprintln!("failed: {}", e),
                Ok(stream) => {
                    thread::spawn(move || {
                        handle_client(stream).unwrap_or_else(|error| eprintln!("{:?}", error));
                    });
                }
            }
        }
    } else if args[1] == "--client" {
        let mut stream = TcpStream::connect("127.0.0.1:8888").expect("Could not connect to server");
        println!("Please provide a 3D point as three comma separated integers");
        loop {
            let mut input = String::new();
            let mut buffer: Vec<u8> = Vec::new();
            stdin()
                .read_line(&mut input)
                .expect("Failed to read from stdin");
            let parts: Vec<&str> = input.trim_matches('\n').split(',').collect();
            let point = Point3D {
                x: parts[0].parse().unwrap(),
                y: parts[1].parse().unwrap(),
                z: parts[2].parse().unwrap(),
            };
            stream
                .write(serde_json::to_string(&point).unwrap().as_bytes())
                .expect("Failed to write to server");

            let mut reader = BufReader::new(&stream);

            reader
                .read_until(b'\n', &mut buffer)
                .expect("Could not read into buffer");
            print!(
                "{}",
                str::from_utf8(&buffer).expect("Could not write buffer as string")
            );
        }
    }
}

How do I know what length of buffer to allocate before reading in the string? If my buffer is too large, serde fails to deserialize it with an error saying that there are invalid characters. Is there a better way to do this?

I did not know that existed. I tried this let input: Point3D = serde_json::from_reader(&stream)?; and looks like it just waits for a EOF from the client. How do I make from_reader exit on a special character from the client? — Abhishek Chanda
Looking at the example here docs.serde.rs/serde_json/fn.from_reader.html I thought from_reader on the file's reader exits when it sees a EOF, a special character in the case of reading from a file. I was wondering if I need to treat newline as a special character and make from_reader return on getting a newline. Or am I misunderstanding this? — Abhishek Chanda
An EOF is not a "character". You never answered the question: how do you, the programmer, know when the socket is "done"? You are defining a protocol, but you haven't actually shared what your protocol is, so we can't tell you how to implement it. Do newlines have something to do with the data that is coming over the network? — Shepmaster
I am trying to use a newline from the client to the server to mark that the socket is done. Sorry for being vague. Added my server and client codes for some context. — Abhishek Chanda
You seem to already know about .read_until — why is that not valid to use here? — Shepmaster

Shepmaster Shepmaster · Accepted Answer · 2017-11-06T22:49:12

Place the TcpStream into a BufReader. This allows you to read until a specific byte (in this case a newline). You can then parse the read bytes with Serde:

use std::io::{BufRead, BufReader};
use std::io::Write;

fn handle_client(mut stream: TcpStream) -> Result<(), Error> {
    let mut data = Vec::new();
    let mut stream = BufReader::new(stream);

    loop {
        data.clear();

        let bytes_read = stream.read_until(b'\n', &mut data)?;
        if bytes_read == 0 {
            return Ok(());
        }

        let input: Point3D = serde_json::from_slice(&data)?;
        let value = input.x.pow(2) + input.y.pow(2) + input.z.pow(2);

        write!(stream.get_mut(), "{}", value)?;
    }
}

I'm being a little fancy by reusing the allocation of data, which means it's very important to reset the buffer at the beginning of each loop. I also avoid allocating memory for the result and just print directly to the output stream.

Deserializing newline-delimited JSON from a socket using Serde

1 Answers