It would help if you provide more details of specific versions of IE and NVDA and whether you are using a keyboard or mouse. I am assuming mouse interaction because that's usually the culprit one in IE.
With that assumption, I don't think it's something you can really fix with just HTML or CSS https://github.com/nvaccess/nvda/issues/5320. NVDA in IE in mouse mode ends up reading a fair amount of unnecessary items. I assume this is what it relates to.
You can try display none, role=presentation, aria-hidden, position off screen, etc. and it will still seem to read it (whether it is inside or outside of the video tag).
So then it comes to why you are wanting this to be hidden. I figure this is not your real code, since you do have sources and they do load. So if the text is there just for older browsers, I would suggest not having the span at all initially. You might want to use modernizr, or do checks to see if the video tag is supported and then if it isn't add in the span. Or if you just want to account for older versions of IE you could put the content within conditional tags.
Unfortunately there doesn't look to be an easy answer from everything that I've found.