0
votes

I need to write a python program that is supposed to be time-efficient. For some logic in the program, I need to load in an array from a pickle dump. The dump size is around 200 MB, so it takes a few seconds to load it into memory, which isn't ideal. I was wondering if there is a way to compile the python program into an executable in such a way that its byte code includes the data from the pickle dump as well so that it doesn't need to read it from the disk on startup?

Alternatively, if a better solution is available to this problem, that would be appreciated.

You will have to read the data from disk anyway, it will just change the format.Klaus D.
Right, it would be loading the program file from disk, and correct me if I'm wrong here, but the computed "EXECUTION" time of the program would be lesser right? (Assuming the time taken to load the program into memory isn't counted as part of execution time). The target is to minimize execution time.dkapur17
How are you actually measuring execution time? Unless it is actually inside the python program, the time to read from disk would be included.ZaydH
If you don't count program load time, then execution would be faster... but don't you have to count program load time? You could have a string holding the data, but then that string will still be in memory for the duration of the execution.tdelaney
I think the problem is that the data has to be in some serialized format. You have a top level object that you pickled and that has to include all of the data in all of the sub objects. If you had code that generated this information, it would likely be much larger and slower.tdelaney