Rust - Multiple Calls to Iterator Methods

1
votes

I have this following rust code:

fn tokenize(line: &str) -> Vec<&str> {
    let mut tokens = Vec::new();
    let mut chars = line.char_indices();
    for (i, c) in chars {
        match c {
            '"' => {
                if let Some(pos) = chars.position(|(_, x)| x == '"') {
                    tokens.push(&line[i..=i+pos]);
                } else {
                    // Not a complete string
                }
            }
            // Other options...
        }
    }
    tokens
}

I am trying to elegantly extract a string surrounded by double quotes from the line, but since chars.position takes a mutable reference and chars is moved into the for loop, I get a compilation error - "value borrowed after move". The compiler suggests borrowing chars in the for loop but this doesn't work because an immutable reference is not an iterator (and a mutable one would cause the original problem where I can't borrow mutably again for position).

I feel like there should be a simple solution to this. Is there an idiomatic way to do this or do I need to regress to appending characters one by one?

2
rustiterator

2 Answers

3
votes

Because a for loop will take ownership of chars (because it calls .into_iter() on it) you can instead manually iterate through chars using a while loop:

fn tokenize(line: &str) -> Vec<&str> {
    let mut tokens = Vec::new();
    let mut chars = line.char_indices();
    while let Some((i, c)) = chars.next() {
        match c {
            '"' => {
                if let Some(pos) = chars.position(|(_, x)| x == '"') {
                    tokens.push(&line[i..=i+pos]);
                } else {
                    // Not a complete string
                }
            }
            // Other options...
        }
    }
}
1
votes

It works if you just desugar the for-loop:

fn tokenize(line: &str) -> Vec<&str> {
    let mut tokens = Vec::new();
    let mut chars = line.char_indices();
    while let Some((i, c)) = chars.next() {
        match c {
            '"' => {
                if let Some(pos) = chars.position(|(_, x)| x == '"') {
                    tokens.push(&line[i..=i+pos]);
                } else {
                    // Not a complete string
                }
            },
            _ => {},
        }
    }
    tokens
}

The normal for-loop prevents additional modification of the iterator because this usually leads to surprising and hard-to-read code. Doing it as a while-loop has no such protection.

If all you want to do is find quoted strings, I would not, however, go with an iterator at all here.

fn tokenize(line: &str) -> Vec<&str> {
    let mut tokens = Vec::new();
    let mut line = line;
    while let Some(pos) = line.find('"') {
        line = &line[(pos+1)..];
        if let Some(end) = line.find('"') {
            tokens.push(&line[..end]);
            line = &line[(end+1)..];
        } else {
            // Not a complete string
        }
    }
    tokens
}