How to partition and use heap memory allocated en masse with Rust?

Question

I've read a lot on Rust lately but am still only beginning to oxidize. My brain retains most of its C/C++ reflexes so pardon me if this question is not relevant because of how things are done in Rust.

Is it generally possible (desirable?) to allocate a block of an arbitrary size on the heap and then map a data structure over it with bindings taking ownership of smaller chunks of memory?

Using C99, one may write this:

typedef unsigned char BYTE;

typedef struct {
    size_t size;
    BYTE payload[];
} flex;

// ...

flex * flex_new(size_t _size) {
    flex *f = malloc(sizeof(flex) + _size * sizeof(BYTE));
    f->size = _size;
    return f;
}

Flexible length arrays may prove useful e.g. when implementing memory pools with chunks of variable size or when allocating memory on the heap dedicated to a thread. The data structure which required size is only known at runtime is allocated in an 'atomic' fashion and its members packed contiguously.

I am wondering whether a similar Rust implementation is possible (without unsafe constructs) and if it is, what it looks like. Maybe the compiler may infer some information using lifetime specifiers in the struct definition?

There are probably good uses for it, but if so, they are likely using unsafe. It doesn't help that many libraries that you'd expect are doing this actually found different ways: Vec is a (ptr, length, capacity) triple rather than a pointer to (length, capacity, items), and Arena piggybacks on Vecs that are never resized. — user395760
I was not aware of Arena and will look into it at once, thanks! I should have stated that I don't care if unsafe is used as long as a safe interface is provided in std or as a candidate for it. — dummydev

Steve Klabnik Steve Klabnik · Accepted Answer · 2014-12-19T20:19:11

In terms of 'flexible length arrays,' you can use Vec::with_capacity to allocate more than your immediate need:

let mut vec: Vec<int> = Vec::with_capacity(10);

// The vector contains no items, even though it has capacity for more
assert_eq!(vec.len(), 0);

// These are all done without reallocating...
for i in range(0i, 10) {
    vec.push(i);
}

// ...but this may make the vector reallocate
vec.push(11);

For the more general case, TypedArena is what you'd want to use, though.

How to partition and use heap memory allocated en masse with Rust?

1 Answers