How can one force Rust to take ownership of memory allocated other than by its safe methods?

Question

In his February 2018 note titled "Memory Safety in Rust: A Case Study with C", Will Crichton wrote:

Rust provides the ability to take ownership of raw pointers, which we do using slice::from_raw_parts_mut and Box::from_raw which tells Rust to treat the memory pointer as a heap-allocated array. After transferring ownership, assuming the memory is valid and of the right size/type, Rust applies its usual memory safety and containment checks.

The relevant part of his code to which the above refers is:

let mut new_data = unsafe {
    let ptr = Heap::default()
        .alloc(Layout::array::<isize>(new_capacity).unwrap())
        .unwrap() as *mut isize;
    Box::from_raw(slice::from_raw_parts_mut(ptr, new_capacity))
};

However, the documentation for Box::from_raw states (emphasis added):

Since the way Box allocates and releases memory is unspecified, the only valid pointer to pass to this function is the one taken from another Box via the Box::into_raw function.

For the avoidance of doubt, the (experimental) Heap API used above to perform memory allocation (since removed in Rust 1.27.0) directly called __rust_alloc in its alloc method—and therefore ptr was not obtained from Box::into_raw.

Is it valid, albeit unsupported, to pass to Box::from_raw raw pointers to freshly allocated memory in order to have Rust take ownership of that memory and enforce its usual safety and containment checks? In particular, will Rust deallocate that memory when the arising Box is destroyed?

If not, how can one force Rust to take such ownership of memory allocated other than by its safe methods?

"If not, how can one force Rust to take such ownership of memory allocated other than by its safe methods?" by definition this make no sense. Also, I will advice to not use Box::from_raw with pointer that doesn't come from Box, first even if it "could be correct in some case" this could go wrong fast, secondly, I doubt that a good API in C will give you the care to release a resource, except some case where you can just call free and I think it's better to call free that to guess if box would have too. — Stargateur

Shepmaster Shepmaster · Accepted Answer · 2019-02-25T00:35:37

Is it valid, albeit unsupported, to pass to Box::from_raw raw pointers to freshly allocated memory

No, it is not valid.

In particular, will Rust deallocate that memory when the arising Box is destroyed?

Yes, and this is the reason why it's invalid.

Memory allocators provide paired allocation and deallocation routines. When you allocate a piece of memory with one allocator, you must free it with that allocator.

If you don't, when the allocator doing the deallocation goes to perform whatever bookkeeping it needs to do, it won't know about that piece of memory. The allocator that actually did the allocation will never mark that memory as unavailable.

These concerns aren't made up, either. I've submitted patches to GLib to correct places where mismatched allocations / deallocations occurred and caused real problems in the wild.

Rust to take such ownership of memory allocated

At the level of raw pointers, ownership is largely a state of mind, just like it is in C or C++. To own something here means that you are responsible for cleaning it up appropriately.

malloc and free are paired allocation/deallocation methods. You could create your own type and implement Drop for it:

use libc::{free, malloc};
use std::{ffi::c_void, mem};

struct MallocBox(*mut i32);

impl MallocBox {
    fn new(v: i32) -> Self {
        unsafe {
            let p = malloc(mem::size_of::<i32>()) as *mut i32;
            *p = v;
            Self(p)
        }
    }
}

impl Drop for MallocBox {
    fn drop(&mut self) {
        unsafe { free(self.0 as *mut c_void) }
    }
}

fn main() {
    MallocBox::new(42);
}

A real implementation would also implement Deref and probably many other traits so that this type is ergonomic to use.

It would be annoying to have to create a MallocBox and JeMallocBox and a MyCustomAllocBox, which is why RFC 1398 proposes a shared trait for allocators. Related work is progressing to convert Box<T> into Box<T, A: Alloc + Default = Global>.

how can one force Rust

There's no concept of "forcing" Rust to do anything, much less when it comes to low level details like this. For example, there's no guarantee that the C code that allocated the pointer doesn't try to free the pointer itself. In an FFI world, ownership is a cooperative agreement.

How can one force Rust to take ownership of memory allocated other than by its safe methods?

1 Answers