GraphicsMagick slower when resizing an image in folder with lots files (+80k)

0
votes

I noticed that when I try to resize an image that resides in a folder with over 80k other images (same level no sub directories) the resize can take almost 2 seconds. (1.92s)

Yet, the same image, in a folder with only 10 other images, is almost instant (0.02s).

  • I'm testing this in batch mode, since my app is using gm4java:1.1.0.
  • Running on windows 10
  • NTFS (I thought this could be an issue, ran contig.exe, but no change)
  • GraphicsMagick 1.3.21

Here are my commands and outputs:

GM> benchmark convert -size 200x200 "C:\lots-of-pics\image399.png[0]" -auto-orient -thumbnail 200x200 "C:\Users\user\AppData\Local\Temp\img-4518761374990603981.png"
Results: 1 threads 1 iter 1.94s user 1.94s total 0.514 iter/s 0.516 iter/cpu
GM> benchmark convert -size 200x200 "C:\less-pics\image399.png[0]" -auto-orient -thumbnail 200x200 "C:\Users\user\AppData\Local\Temp\img-4518761374990603981.png"
Results: 1 threads 1 iter 0.02s user 0.02s total 58.823 iter/s 64.000 iter/cpu

I wasn't able to find anything about this on SO or sourceforge. Any ideas why it's so much slower?

1
windowsperformancegraphicsmagickdisk-io
Have you filed a bug with the GraphicsMagick folks? – Alan
@alan No, wasn't sure it was a bug per say. Is that usually the best way to go about it? – mateuscb
This question doesn't appear to be programming related, with no programming related answer--so my opinion is that the question is off-topic for SO. You have observed unexpected behavior with GM and they would be the best source for resolution for this issue. Filing a bug, along with a reproducible test case should speed that process up. – Alan
You can try writing a simple app that fully reads a given file and benchmark that - if merely reading the file takes that much more time then yes, you have a problem with the filesystem (perhaps do chkdsk?). Otherwise, the library you use must behave differently if there are lots of files next to the one given it to convert - you'll have to investigate that, with a profiler, Sysinternals Process Monitor or via a number of other possible approaches. – cynic
You are surprised that Windows is poor at handling 80,000 files that all start with img-nnnnnnnn.png? – Mark Setchell

1 Answers

0
votes

I should have tried this first. Turns out updating to the latest GraphicsMagick 1.3.24 resolved the issue.

Resizing the same image, regardless of how many other files are in the same folder, now takes the same time.

Looking at the release notes of 1.3.22 this may have fixed it since mentions many files in the directory (I couldn't find the exact commit):

General: Fixed performance problem with sub-image path extraction when there are many files in the directory.