I am sure this is late, but hopefully it helps some one out. It sounds like you may just need to screen the call.
Basically, you can ask the "agent" that you dialed to accept the call and hang up if you do not receive input.
I am not sure what language you are using, but here is a great php/Laravel tutorial to explain:
https://www.twilio.com/docs/tutorials/walkthrough/ivr-screening/php/laravel
The key part is here:
$dialCommand = $response->dial(
['action' => route('agent-voicemail', ['agent' => $agent->id], false),
'method' => 'POST']
);
$dialCommand->number(
$numberToDial,
['url' => route('screen-call', [], false)]
);
Notice that the dial command uses the 'action' to specify a location that is sent a POST request should the call end i.e POST to /agent-voicemail.
Then, the number is dialed along with a 'url' parameter this is the location that will be requested after the agent has picked up but before connecting the two parties.
The /screen-call route then asks the agent to accept the call, if no input is received it hangs up and will make a POST request to the initial setup /agent-voicemail route.
This method will handle your scenario because if it goes to voicemail no input will be received and the call will end.