Can you control borrowing a struct vs borrowing a field?

Question

I'm working on a program involving a struct along these lines:

struct App {
    data: Vec<u8>,
    overlay: Vec<(usize, Vec<u8>)>,
    sink: Sink,
}

In brief the data field holds some bytes and overlay is a series of byte sequences to be inserted at specific indices. The Sink type is unimportant except that it has a function like:

impl Sink {
    fn process<'a>(&mut self, input: Vec<&'a [u8]>) {
        // ...
    }
}

I've implemented an iterator to merge the information from data and overlay for consumption by Sink.

struct MergeIter<'a, 'b> {
    data: &'a Vec<u8>,
    overlay: &'b Vec<(usize, Vec<u8>)>,
    // iterator state etc.
}

impl<'a, 'b> Iterator for MergeIter<'a, 'b> {
    type Item = &'a [u8];
    // ...
}

This is I think a slight lie, because the lifetime of each &[u8] returned by the iterator isn't always that of the original data. The data inserted from overlay has a different lifetime, but I don't see how I can annotate this more accurately. Anyway, the borrow checker doesn't seem to mind - the following approach works:

fn merge<'a, 'b>(data: &'a Vec<u8>, overlay: &'b Vec<(usize, Vec<u8>)>, start: usize) -> Vec<&'a [u8]> {
    MergeIter::new(data, overlay, start).collect()
}

impl App {
    fn process(&mut self) {
        let merged = merge(&self.data, &self.overlay, 0);
        // inspect contents of 'merged'
        self.sink.process(merged);
    }
}

I end up using this merge function all over the place, but always against the same data/overlay. So I figure I'll add an App::merge function for convenience, and here's where the problem begins:

impl App {
    fn merge<'a>(&'a self, start: usize) -> Vec<&'a [u8]> {
        MergeIter::new(&self.data, &self.overlay, start).collect()
    }

    fn process(&mut self) {
        let merged = self.merge(0);
        // inspect contents of 'merged'
        self.sink.process(merged);
    }
}

App::process now fails to pass the borrow checker - it refuses to allow the mutable borrow of self.sink while self is borrowed.

I've wrestled with this for some time, and if I've understood correctly the problem isn't with process but with this signature:

fn merge<'a>(&'a self, start: usize) -> Vec<&'a [u8]> {

Here I've essentially told the borrow checker that the references returned in the vector are equivalent to the self borrow.

Even though I feel like I've now understood the problem, I still feel like my hands are tied. Leaving the lifetime annotations out doesn't help (because the compiler does the equivalent?), and with only the two references involved there's no way I can see to tell rust that the output reference has a lifetime bound to something else.

I also tried this:

fn merge<'a, 'b>(&'b self, start: usize) -> Vec<&'a [u8]> {
    let data: &'a Vec<u8> = &self.data;
    MergeIter::new(&self.data, &self.overlay, start).collect()
}

but the compiler complains about the let statement ("unable to infer appropriate lifetime due to conflicting requirements" -- I also find it infuriating that the compiler doesn't explain said requirements).

Is it possible to achieve this? The Rust Reference is kind of light on lifetime annotations and associated syntax.

rustc 1.0.0-nightly (706be5ba1 2015-02-05 23:14:28 +0000)

Your last attempt is incorrect - you're essentially saying that 'a can be arbitrary lifetime the caller chooses, so it may as well chose 'static. But your code does not return a reference with static lifetime, it returns a reference with lifetime of self.data. This can't possibly work, so such signature is considered unsound. And BTW, the compiler usually shows the conflicting requirements - it highlights scopes with which conflicting lifetimes are associated. The rest of your question requires more research. — Vladimir Matveev
&Vec<T> could be made a tiny bit more efficient by changing it to &[T]; &Vec<T> will automatically coerce to &[T] also, so there is no increased burden in using it. — Chris Morgan
@VladimirMatveev I've seen scopes highlighted for some lifetime errors, but not in this particular case. Which just makes it all the more frustrating :P — sqweek

Vladimir Matveev Vladimir Matveev · Accepted Answer · 2015-04-27T13:24:48

Yes, you've guessed correctly - the error happens because when you have merge method accept &self, the compiler can't know at its call site that it uses only some fields - merge signature only tells it that the data it returns is somehow derived from self, but it doesn't tell how - and so the compiler assumes the "worst" case and prevents you from accessing other fields self has.

I'm afraid there is no way to fix this at the moment, and I'm not sure there ever will be any. However, you can use macros to shorten merge invocations:

macro_rules! merge {
    ($this:ident, $start:expr) => {
        MergeIter::new(&$this.data, &$this.overlay, $start).collect()
    }
}

fn process(&mut self) {
    let merged = merge!(self, 0);
    // inspect contents of 'merged'
    self.sink.process(merged);
}

Can you control borrowing a struct vs borrowing a field?

2 Answers