"Variable does not live long enough" when returning a Result containing a reference but it does live long enough

Question

I'm implementing a small utility and the compiler is telling me that a variable (a TcpStream) does not live long enough and is advising me to find a way to make it live exactly as long as it is currently living.

Error message

error[E0597]: `stream` does not live long enough
  --> src/main.rs:47:35
   |
47 |         match handle_request(&mut stream){
   |                                   ^^^^^^ borrowed value does not live long enough
...
54 | }
   | - borrowed value only lives until here
   |
note: borrowed value must be valid for the anonymous lifetime #1 defined on the function body at 43:1...
  --> src/main.rs:43:1
   |
43 | / fn handle_array(stream: &mut BufReader<TcpStream>) -> Result<Data,Errors>
44 | | {
45 | |     let mut array: Vec<Data> = Vec::with_capacity(50);//arbitrary size, picked differently in the complete program
46 | |     for _x in 0..50 {
...  |
53 | |     Ok(Data::Array(array))
54 | | }
   | |_^

Code

Rust playground snippet with the exact problem

use std::collections::HashMap;
use std::io::BufReader;
use std::io::Read;
use std::net::TcpStream;

static TOKEN: &[u8; 2] = b"\r\n";

fn main() {}

#[derive(Debug, Clone)]
pub enum Data {
    String(Vec<u8>),
    Error(Vec<u8>),
    Integer(i64),
    Binary(Vec<u8>),
    Array(Vec<Data>),
    Dictionary(HashMap<String, Data>),
}

#[derive(Debug, Clone)]
pub enum Errors<'a> {
    CommandError(&'a str),
    EndOfConnection,
    NotImplemented,
}

pub fn handle_request(stream: &mut BufReader<TcpStream>) -> Result<Data, Errors> {
    //TODO handle the empty stream
    let mut buff: [u8; 1] = *b"0";
    stream.read_exact(&mut buff); //TODO: handle error here
    match &buff {
        /* part skipped, not relevant */
        b"*" => handle_array(stream),
        &[_] => Err(Errors::CommandError("Bad request")),
    }
}

/*part skipped, not relevant  */

fn handle_array(stream: &mut BufReader<TcpStream>) -> Result<Data, Errors> {
    let mut array: Vec<Data> = Vec::with_capacity(50); //arbitrary size, picked differently in the complete program
    for _x in 0..50 {
        match handle_request(&mut stream) {
            Ok(x) => array.push(x.clone()),
            Err(x) => return Err(x.clone()),
        }
    }

    Ok(Data::Array(array))
}

I'm really stuck on this one.

It seems that I can't use the value of the Err. If I replace

match handle_request(&mut stream){
    Ok(x) => array.push(x.clone()),
    Err(x) => return Err(x.clone()),
}

with

match handle_request(&mut stream){
    Ok(x) => array.push(x.clone()),
    Err(_) => return Err(Errors::NotImplemented),
}

the problem is solved, but I don't know why.

Answer 1

Your problem can be reduced to this:

struct Reader;
struct Data;
struct Errors<'a>(&'a str);

fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
    for _ in 0..0 {
        handle_request(&mut stream)?;
    }

    unimplemented!();
}

fn handle_request(_stream: &mut Reader) -> Result<Data, Errors> {
    unimplemented!()
}

fn main() {}

error[E0597]: `stream` does not live long enough
  --> src/main.rs:7:29
   |
7  |         handle_request(&mut stream)?;
   |                             ^^^^^^ borrowed value does not live long enough
...
11 | }
   | - borrowed value only lives until here
   |
note: borrowed value must be valid for the anonymous lifetime #1 defined on the function body at 5:1...
  --> src/main.rs:5:1
   |
5  | / fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
6  | |     for _ in 0..0 {
7  | |         handle_request(&mut stream)?;
8  | |     }
9  | |
10 | |     unimplemented!();
11 | | }
   | |_^

In the body of handle_array, stream is of type &mut Reader. However, when calling handle_request, you take another reference to it, creating a &mut &mut Reader.

Adding some explicit lifetimes to the code (for educational purposes, this doesn't compile), the code would look kind of like this:

fn handle_array<'stream>(stream: &'stream mut Reader) -> Result<Data, Errors> {
    let tmp: &'tmp mut &'stream mut Reader = &mut stream;
    if let Err(x) = handle_request(tmp)

handle_request requires a &mut Reader, so the compiler inserts some code to align these two types for you. The compiler has to be conservative about how it performs this conversion, so it picks the shorter lifetime:

fn handle_array<'stream>(stream: &'stream mut Reader) -> Result<Data, Errors> {
    let tmp: &'tmp mut &'stream mut Reader = &mut stream;
    let tmp2: &'tmp mut Reader = tmp;
    if let Err(x) = handle_request(tmp2)

The next aspect of the problem is that lifetime elision has kicked in for both functions. Their expanded forms look like:

fn handle_array<'stream>(stream: &'stream mut Reader) -> Result<Data, Errors<'stream>>

fn handle_request<'_stream>(_stream: &_stream mut Reader) -> Result<Data, Errors<'_stream>>

This means that the lifetime of the returned Errors is tied to the lifetime of the argument, but in your case, the argument to handle_request has the shorter 'tmp lifetime, not the lifetime of 'stream. This shows why you get the compiler error: You are trying to return an Errors that can only live inside the function (the lifetime of the variable stream itself), but you are trying to return a reference that needs to live longer.

We can solve this by only passing stream to handle_request:

handle_request(stream)?;

Unfortunately, this only changes the error:

error[E0499]: cannot borrow `*stream` as mutable more than once at a time
  --> src/main.rs:9:40
   |
9  |         if let Err(x) = handle_request(stream) {
   |                                        ^^^^^^ mutable borrow starts here in previous iteration of loop
...
15 | }
   | - mutable borrow ends here

This part is much harder to explain. See:

• "Problem Case #3" of the Non-Lexical Lifetimes RFC (RFC 2094)
• Cannot borrow `x` as mutable more than once at a time
• Cannot borrow node as mutable more than once while implementing a binary search tree

This is a really rough edge of Rust's right now, but it's getting closer and closer to being fixed! Right now, however, you have two main choices:

Call the function twice

This probably won't work because you can't read from the stream twice, but in other cases it might be useful:

fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
    let mut array = vec![];
    for _ in 0..0 {
        if handle_request(stream).is_err() {
            return handle_request(stream);
        }
        if let Ok(r) = handle_request(stream) {
            array.push(r);
        };
    }

    unimplemented!();
}

Remove the references

Give up on trying to have references in this case for now.

struct Errors(String);

fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
    let mut array = vec![];
    for _ in 0..0 {
        array.push(handle_request(stream)?);
    }

    unimplemented!();
}

Which I'd write using iterators for efficiency:

fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
    let array = (0..0)
        .map(|_| handle_request(stream))
        .collect::<Result<Vec<_>, _>>()?;

    unimplemented!();
}

The future?

With the unstable NLL feature and the experimental "Polonius" implementation, this code works:

struct Errors<'a>(&'a str);

fn handle_array(stream: &mut Reader) -> Result<Data, Errors> {
    let mut array = vec![];
    for _ in (0..0) {
        array.push(handle_request(stream)?);
    }

    unimplemented!();
}

It will just be a while before this is generally available...