I have finally found the answer to the issue. All said and done, the issue was that the interop assemblies were framework version 4.0 instead of framework version 2.0. Visual Studio created them this way for some reason, so I had to change them by using Visual Studio 2008 to create new versions of them.
The process to find this out (which most of is already stated above, but repeated here for helpfulness) was to place all my VB.NET dlls and the interop assembly dlls in the directory of the VB6 exe. (I created sub folders for each one for easier cleanup later.) This still resulted in the same error, but the assembly builder log showed a different reason for the error. This time it was able to load the assembly without issue, but was not able to complete the setup of the assembly.
Researching this led me to the idea of checking the framework versions of the interop assemblies using ILSpy to find out they are version 4.0 while my dlls are version 2.0.
Initially I tried recreating them in Visual Studio 2017, since my project uses framework 2.0, but it still created them as framework 4.0 for some reason. I then tried in Visual Studio 2008, and it was able to create the correct framework 2.0 versions of the interop assemblies, and now everything works as expected.