Just reading the documentation from MSDN,
If the UnhandledException event is handled in the default application domain, it is raised there for any unhandled exception in any thread, no matter what application domain the thread started in. If the thread started in an application domain that has an event handler for UnhandledException, the event is raised in that application domain. If that application domain is not the default application domain, and there is also an event handler in the default application domain, the event is raised in both application domains.
For example, suppose a thread starts in application domain "AD1", calls a method in application domain "AD2", and from there calls a method in application domain "AD3", where it throws an exception. The first application domain in which the UnhandledException event can be raised is "AD1". If that application domain is not the default application domain, the event can also be raised in the default application domain.
So the solution is create a seperate domain for my component and create the event handler for my domain
AppDomain domain = AppDomain.CreateDomain("MyDomain");
domain.UnhandledException += MyDomainUnhandledException
static void CurrentDomainUnhandledException(objectsender,UnhandledExceptionEventArgs e)
{
//Exception handling logic for my domain
}
This will make sure that the exceptions do not propogate to the parent application