I know for Power BI that you don't require the On-Prem Gateway to refresh data for Azure SQL Database.
Surprisingly I'm struggling to find information about Amazon RDS. Do you require the On-Prem Gateway to refresh data in the Power BI Service for Amazon RDS?