Provided you still have access to the original MFA device, or originally configured to also allow SMS MFA login, these instructions worked for me. This is based on what @Carl linked to above (http://www.uclabs.blog/2018/03/mfa-with-guest-access-and-different.html), but expanded out a bit as I struggled to follow it as written.
BTW I recommend doing all this in a private/incognito window, to be sure you know what you are logged in as.
Login to https://myapplications.microsoft.com/ using your 'normal' tenancy credentials.
Select the profile badge for you (circle, top right), and select 'Switch organisation' to log into the guest tenancy you want to reconfigure. At this point if you don't have access to the current MFA authenticator device you will need to use 'login another way' to use SMS MFA for this login.
Now, in the guest tenancy, select your badge again, and select 'My Profile'. If you don't see 'My Profile', use the ellipsis (...) and select to leave the 'new experience'. When the page reloads, now you should find the 'My Profile' link under your badge.
On the profile page, right hand side, you should see 'Additional Security Verification'. This should get you to this page in the guest tenancy:
https://account.activedirectory.windowsazure.com/Proofup.aspx
From there you should see options to (re)setup your Authenticator app (scan the QR code etc...). Don't forget to delete the registration for your old phone too.