Shared circular references in Rust

Question

I'm trying to implement a very naive Thread Pool model. For now the responsibilities of the thread pool are:

create a new thread and return a reference to it
Maintain a map of created threads
Mutate threads according to different events triggered by threads

My main problem is that the requirement above forces me to let the thread pool keep a HashMap<Id, Thread> of threads, but also provide a reference to threads when they are created.

On top of that, I need to be able to call methods of the thread pool from inside any thread that effectively end up mutating 1 or more threads (the caller and the targets).

Here is a non functional implementation:

use std::collections::HashMap;
use std::cell::RefCell;

type Id = u32;
type ThreadPoolRef = RefCell<ThreadPool>;

#[derive(Debug)]
struct ThreadPool {
    pool: HashMap<Id, RefCell<Thread>>,
    id_count: Id
}

impl ThreadPool {
    fn new() -> ThreadPool {
        ThreadPool {
            pool: HashMap::new(),
            id_count: 1
        }
    }

    fn create(&mut self) -> &RefCell<Thread> {
        let thread: RefCell<Thread> =
            RefCell::new(
                Thread::new(self.id_count, RefCell::new(self))
            );
        self.id_count = self.id_count + 1;
        self.pool.insert(self.id_count, thread);
        self.pool.get(&self.id_count).unwrap()
    }
}

#[derive(Debug, Clone)]
struct Thread {
    id: Id,
    pool: ThreadPoolRef
}

impl Thread {
    fn new(id: Id, pool: ThreadPoolRef) -> Thread {
        Thread {
            id: id,
            pool: pool
        }
    }
}

fn main() {
    let thread_pool = ThreadPool::new();
    let thread1 = thread_pool.create();
    let thread2 = thread_pool.create();
    // The final goal is to call thread1.method()
    // that mutates thread1 and calls thread_pool.method2()
    // which in turn will call thread2.method3() that will effectively
    // mutate thread 2
}

Rust Play

I tried several things, like the use of RefCell but I started to get a lot of lifetime parameters missing errors.

This is a stripped-down version that I expect to be the most simple to explain.

malbarbo malbarbo · Accepted Answer · 2016-05-29T01:40:09

I need to be able to call methods of the thread pool from inside any thread

This requires that the thread pool data be in a mutual-exclusion mechanism, like Mutex or RwLock (RefCell is not appropriate for multi-threaded situations, see the book). Besides, each thread must keep a reference to the thread pool data, as the thread pool stores the threads, this creates a problem. To solve this problem, we can put the thread pool data in an Arc and store a Weak reference to it in each thread. Note that we use weak references to avoid reference cycles.

that effectively end up mutating 1 or more threads (the caller and the targets).

This requires that the thread data be in a mutual-exclusion mechanism. To finalize the requirements, note that as the thread pool data is in a Mutex, we cannot return references to threads (that would require keeping the thread pool data locked), so we also put thread data in a Arc.

Here is an example of implementation using Mutex (Playground):

use std::collections::HashMap;
use std::sync::{Arc, Weak, Mutex};

type Id = u32;

struct ThreadPool {
    inner: Arc<Mutex<ThreadPoolInner>>,
}

struct ThreadPoolInner {
    pool: HashMap<Id, Arc<Mutex<ThreadInner>>>,
    id_count: Id,
}

impl ThreadPool {
    fn new() -> ThreadPool {
        let inner = ThreadPoolInner {
            pool: HashMap::new(),
            id_count: 0,
        };
        ThreadPool { inner: Arc::new(Mutex::new(inner)) }
    }

    fn create(&self) -> Thread {
        let mut inner = self.inner.lock().unwrap();
        let thread = Thread {
            inner: Arc::new(Mutex::new(ThreadInner {
                id: inner.id_count,
                pool: Arc::downgrade(&self.inner),
            })),
        };
        inner.id_count += 1;
        let id = inner.id_count;
        inner.pool.insert(id, thread.inner.clone());
        thread
    }

    fn get(&self, id: Id) -> Option<Thread> {
        let inner = self.inner.lock().unwrap();
        inner.pool.get(&id).map(|t| Thread { inner: t.clone() })
    }

    fn some_mut_method(&self) {
        let _inner = self.inner.lock().unwrap();
        println!("something with pool");
    }
}

struct Thread {
    inner: Arc<Mutex<ThreadInner>>,
}

impl Thread {
    fn get_pool(&self) -> Option<ThreadPool> {
        let inner = self.inner.lock().unwrap();
        // pool is a weak reference, upgrate try to get an Arc from it
        inner.pool.upgrade().map(|inner| ThreadPool { inner: inner })
    }

    fn some_mut_method(&self) {
        if let Some(pool) = self.get_pool() {
            pool.some_mut_method();
            let _t2 = pool.get(2).unwrap();
            println!("something with t2");
        }
    }
}

#[derive(Clone)]
struct ThreadInner {
    id: Id,
    pool: Weak<Mutex<ThreadPoolInner>>,
}

fn main() {
    let pool = ThreadPool::new();
    let t1 = pool.create();
    let _t2 = pool.create();
    t1.some_mut_method();
}

Shared circular references in Rust

1 Answers