Cause
The problem is that AAC is a streamed format. This means that no global file header is provided as with a container format (for example like "mp4"), so the browser has to guess the time based on bit-rate from the initially loaded packet headers (ADTS) and file length provided by the server.
Since the browser doesn't know the actual sample length until the entire stream is consumed the result will vary depending on how the browser (or the underlying system) try to predict the duration time.
AAC can also be encoded with variable bit rate (VBR) which makes it very hard to correctly predict the duration since the bit-rate found at the beginning may not be the bit-rate used for other sample packets later on.
There is of course also the chance that the file/encoding is corrupt or faulty.
These same challenges also applies to MP3 files btw. which is why they sometimes are shown with an incorrect duration as well.
Workarounds
The first is to encode the AAC using constant bit-rate (CBR). This will allow the browser to predict the time more reliable as VBR can throw off the calculations at any point if the variations are noticable. This will most likely affect the file size.
If CBR is not an option, one workaround is to supply a container format for the AAC stream such as MP4 which can store the duration in the global header which is loaded at the beginning - MP4s can be consumed as buffered audio by the Audio element. When doing this you will get the correct time (with perhaps some rounding error for the minor digit):
AAC loaded in MP4 container in Firefox:

AAC loaded in MP4 container in Chrome

To produce a MP4 container for AAC you can use for example the free ffmpeg with these arguments which will copy the AAC as-is (no re-encoding takes place so the result will be of the same quality and bit-rate as before):
ffmpeg.exe -i "Lesson+53-A.aac" -c:a copy "Lesson+53-A.mp4"
you will also see ffmpeg point out this issue:
[aac @ 0000000001c624a0] Estimating duration from bitrate, this may be inaccurate [...]
Another perhaps less convenient workaround is to provide the duration as metadata in for example the filename itself (something like "myfile_MMSS.aac") or as a hash-tag/argument in the url, or side-load the metadata separately.
This latter will have certain implications though as we can not override the native player's duration field (in a cross-browser friendly way) which may require you to build a custom player interface so you can show the metadata as duration instead of the browser predicted one.