I recommend you break these up into single commands until you have a good understanding of what they do and how they work.
This command as written gets every user/shared mailbox.
Get-Mailbox -RecipientTypeDetails UserMailbox,SharedMailbox -ResultSize Unlimited
You're gathering every mailbox each iteration of the loop.
This command would get permissions for a single mailbox if the mailbox is piped into it or passed in through parameter
$mailbox | Get-MailboxPermission
or
Get-MailboxPermission -Identity $mailbox.identity
Adding the user would get mailbox permissions that are granted or denied for the specified user, should any exist. There is a good chance that you would be querying the same mailbox many times for different users.
I am not aware of a way to take a user and find all mailboxes they have rights granted/denied without iterating all mailboxes. So I recommend we first get all mailboxes and their individual rights (both denied or not, and you could further filter for either/or.) The Identity is the canonical name, I'm guessing you would prefer the email address of the mailbox.
# To keep the command line smaller we'll save the filter in a variable
$wherefilter = {$_.user -ne "NT AUTHORITY\SELF" -and $_.isinherited -eq $false}
# Get all mailboxes and each right assigned to them, stored as a hashtable
Get-Mailbox -RecipientTypeDetails usermailbox,sharedmailbox -ResultSize Unlimited | ForEach-Object {
foreach($permission in Get-MailboxPermission $_.identity | Where-Object $wherefilter)
{
[PSCustomObject]@{
Mailbox = $_.PrimarySMTPAddress
User = $permission.user
Type = $_.recipienttype
AccessRights = $($permission.accessrights)
Deny = $permission.deny
}
}
} -OutVariable permissions
By using the -OutVariable parameter, we get to see the data and store in a variable at once. The output will be objects that look like this

Finally, If you create a hash table (lookup table) with this data it can simplify the final part of gathering all permissions per user. The Group-Object command has a handy -AsHashTable parameter that we can take advantage of.
$permissions = $permissions | Group-Object -Property {$_.user.split('\')[1]} -AsHashTable -AsString
Now you can take your list of users and with their SamAccountName retrieve any permission they have using dot notation
Import-CSV $outfile |
ForEach-Object {$permissions.($_.samaccountname)} |
Export-Csv $outbox -NoTypeInformation
One last note - you only want to use the Format-* cmdlets for console output. Do not try to pipe them to other cmdlets such as Export-Csv as the formatted output is no longer well formed objects.