RDC stands for "radio duty cycling". The RDC layer uses radio functions directly, so sending a packet could be as simple as calling NETSTACK_RADIO.send(packetbuf_hdrptr(), packetbuf_totlen()).
The Contiki network stack has this layering structure:
NETWORK layer -> MAC layer -> RDC layer -> RADIO layer.
(In recent versions there's an additional LLSEC layer between NETWORK and MAC layers.)
So an implementation of RDC layer API uses the RADIO layer API and is called by / reports to MAC layer API. You can look at core/net/mac/nullrdc.c to see how to implement a simple RDC driver.
As you noted, NETSTACK_RDC.send(mac_callback_t sent, void *ptr) takes two parameters. The first is of callback type which you need to call after finishing the sending, the second is user data you need to pass to the to the callback.
The callback is typedef in net/mac/mac.h as:
typedef void (* mac_callback_t)(void *ptr, int status, int transmissions);
ptr is the user data pointer passed to NETSTACK_RDC.send;
status is MAC status code, such as MAC_TX_OK - the result of the
transmission;
transmissions is the number of transmissions.