SPFX (teams), Graph API on Behalf of in the backend

Question

I have a webpart SPFX hosted on teams. The webpart is fairly complex and has therefore a corresponding backend.

When the user performs a specific search action in the frontend, I want the backend to merge some data from a database and some data from graph, before returning a response to the frontend.

So the flow is something like this:

1. User logs into teams.
2. User inputs search term into search field (frontend).
3. frontend needs an access token (Which is generated by teams, based on the signed in user) to access the backend.

Here is how the access tokens are generated in the frontend (spfx webpart):

let provider = await context.aadTokenProviderFactory.getTokenProvider();

let graphToken = await provider.getToken("https://graph.microsoft.com/", true);

let backendToken = await provider.getToken("backend base url", true);

My problem is in the backend, I have not been able to create a access token that gives me access to graph api on behalf of the user which is logged onto teams.

I can generate a token like this no problems:

IConfidentialClientApplication confidentialClientApplication = ConfidentialClientApplicationBuilder
                    .Create(clientId)
                    .WithTenantId(tenantId)
                    .WithClientSecret(clientSecret)
                    .Build();

                ClientCredentialProvider authProvider = new ClientCredentialProvider(confidentialClientApplication);
                return new GraphServiceClient(authProvider);

But this client is based on client id and secret and not on behalf of the user logged onto teams.

Is there a way to do what I am trying to achieve? without requiring that the users logs onto teams (1) and then logs onto the backend(2).