OutOfMemoryException loading a large XML file with F# TypeProvider

Question

For a small project I am trying to use the F# Type Providers in a Visual Studio 2013 solution.

Specifically I am trying to load an XML file, which is roughly 1.5GB, with the FSharp.Data.XmlProvider. After the program has used around 3GB's of ram I get a System.OutOfMemoryException.

I have tried to set all the preferences I can find in the project settings to x64 and disabled 'prefer 32-bit'

The FSharp.Data.CsvProvider has a cache option which makes it able to handle big datasets. However I have not found a similar option for the XmlProvider.

Has anyone had any success loading large XML files with the XmlProvider?

Answer 1

You'll have the same problem using plain XElement on a file that big. It's not a supported scenario, you'll have to use a SAX-like API like XmlReader for that