How to write trait & impl with lifetimes for iterators?

Question

I'm trying to understand how to write a trait and an impl for it for my own types that will process some input data. I'm starting with a simple example where I want to process the input 1, 2, 3, 4 with a trait Processor. One implementation will skip the first element and double all remaining inputs. It should therefore look like this:

trait Processor {} // TBD

struct SkipOneTimesTwo;
impl Processor for SkipOneTimesTwo {} // TBD

let numbers = vec![1, 2, 3, 4];
let it = numbers.iter();
let it = Box::new(it);

let proc = SkipOneTimesTwo;
let four_to_eight = proc.process(it);
assert_eq!(Some(4), four_to_eight.next());
assert_eq!(Some(6), four_to_eight.next());
assert_eq!(Some(8), four_to_eight.next());
assert_eq!(None, four_to_eight.next());

So my assumption is that my trait and the corresponding implementation would look like this:

trait Processor {
    // Arbitrarily convert from `i32` to `u32`
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>>;
}

struct SkipOneTimesTwo;
impl Processor for SkipOneTimesTwo {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>> {
        let p = it.skip(1).map(|i| 2 * (i as u32));
        Box::new(p)
    }
}

This code doesn't work as-is. I get the following error:

7 |     let four_to_eight = proc.process(it);
  |                                      ^^ expected `i32`, found reference
  |
  = note:   expected type `i32`
          found reference `&{integer}`
  = note: required for the cast to the object type `dyn Iterator<Item = i32>`

If my input data were very large, I wouldn't want the entire dataset to be kept in-memory (the whole point of using Iterator), so I assume that using Iterator<T> should stream data through from the original source of input until it is eventually aggregated or otherwise handled. I don't know what this means, however, in terms of what lifetimes I need to annotate here.

Eventually, my Processor may hold some intermediate data from the input (eg, for a running average calculation), so I will probably have to specify a lifetime on my struct.

Working with some of the compiler errors, I've tried adding 'a, 'static, and '_ lifetimes to my dyn Iterator<...>, but I can't quite figure out how to pass along an input iterator and modify the values lazily.

Is this even a reasonable approach? I could probably store the input Iterator<Item = i32> in my struct and impl Iterator<Item = u32> for SkipOneTimesTwo, but then I would presumably lose some of the abstraction of being able to pass around the Processor trait.

pretzelhammer pretzelhammer · Accepted Answer · 2021-01-18T20:51:44

All iterators in Rust are lazy. Also, you don't need to use lifetimes, just use into_iter() instead of iter() and your code compiles:

trait Processor {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>>;
}

struct SkipOneTimesTwo;

impl Processor for SkipOneTimesTwo {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>> {
        let p = it.skip(1).map(|i| 2 * (i as u32));
        Box::new(p)
    }
}

fn main() {
    let numbers = vec![1, 2, 3, 4];
    let it = numbers.into_iter(); // only change here
    let it = Box::new(it);
    
    let pro = SkipOneTimesTwo;
    let mut four_to_eight = pro.process(it);
    assert_eq!(Some(4), four_to_eight.next());
    assert_eq!(Some(6), four_to_eight.next());
    assert_eq!(Some(8), four_to_eight.next());
    assert_eq!(None, four_to_eight.next());
}

playground

How to write trait & impl with lifetimes for iterators?

1 Answers