Clone failing at large depths

1
votes

I am having trouble cloning the FFmpeg repo. Using a binary search algorithm, I think I narrowed the issue around a particular depth. Notice the inconsistent results

$ git clone --depth 916 git://source.ffmpeg.org/ffmpeg
Cloning into 'ffmpeg'...
remote: Counting objects: 16737, done.
remote: Compressing objects: 100% (8454/8454), done.
remote: Total 16737 (delta 11293), reused 11481 (delta 8105)
Receiving objects: 100% (16737/16737), 11.32 MiB | 398.00 KiB/s, done.
Resolving deltas: 100% (11293/11293), done.

$ git clone --depth 916 git://source.ffmpeg.org/ffmpeg
Cloning into 'ffmpeg'...
remote: Counting objects: 16737, done.
remote: Compressing objects: 100% (8454/8454), done.
remote: Total 16737 (delta 11291), reused 11482 (delta 8105)
Receiving objects: 100% (16737/16737), 11.32 MiB | 390.00 KiB/s, done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failed

$ git clone --depth 916 git://source.ffmpeg.org/ffmpeg
Cloning into 'ffmpeg'...
remote: Counting objects: 16737, done.
remote: Compressing objects: 100% (8454/8454), done.
remote: Total 16737 (delta 11290), reused 11481 (delta 8105)
Receiving objects: 100% (16737/16737), 11.32 MiB | 401.00 KiB/s, done.
Resolving deltas: 100% (11290/11290), done.
fatal: missing blob object 'e893922133e1837d51077b07b6eb2ef3d5f269ec'
fatal: remote did not send all necessary objects

$ git clone --depth 916 git://source.ffmpeg.org/ffmpeg
Cloning into 'ffmpeg'...
remote: Counting objects: 16737, done.
remote: Compressing objects: 100% (8454/8454), done.
remote: Total 16737 (delta 11292), reused 11481 (delta 8105)
Receiving objects: 100% (16737/16737), 11.32 MiB | 394.00 KiB/s, done.
Resolving deltas: 100% (11292/11292), done.
Checking out files: 100% (3637/3637), done.

How do I fix this issue so that I can clone at this and full depth?

$ git --version
git version 1.8.3.1
2
gitgit-clone
I had no trouble with either git clone git://source.ffmpeg.org/ffmpeg or git clone --depth 916 git://source.ffmpeg.org/ffmpeg. Are you still having the trouble? What version of git? What OS? – Christopher
Failing hardware for either you or the server? The second and third look like corruption. It doesn't really look like the number of commits is key here given the different failures. Try a git clone -v and maybe we'll get more information. – Peter Lundgren

2 Answers

2
votes

In this case the issue is caused by using latest Cygwin (1.7.21) with latest Git from Cygwin Ports (1.8.3.1).

Workaround is to use Adam Dinwoodie’s build

wget tastycake.net/~adam/cygwin/x86/git/git-1.8.5.2-1.tar.xz
tar -x -C / -f git-1.8.5.2-1.tar.xz
1
votes

This has been annoying me for months, and I just now found a workaround that seems to be working for me.

I noticed that every time I compiled git from source under Cygwin and tried to check out a large repo (~1GB repo with an additional 1GB of submodules) that I would fail nearly 100% of the time with this error. However, it always worked if I used the precompiled git from Cygwin's setup.exe. Even if I compiled that same version (1.7.9) myself, my compile would fail and their compile worked. I diff'd an objdump on both .exe files, and the only significant difference was that my .exe used cygcrypto-1.0.0.dll, and they used cygcrypto-0.9.8.dll.

So, I backed up cygcrypto-1.0.0.dll and symlinked cygcrypto-1.0.0.dll to 0.9.8. I have now successfully checked out the repo and submodules successfully 3 times in a row (vs the previous near 100% failure rate). I can't guarantee this fixes the problem 100%, but it looks very very promising.

I'm assuming if you are using Git, then you probably have an idea of the possible issues with symlinking a new DLL version to an old version. Use at your own risk.

(Bonus points if someone replies with how to get my git build to link against 0.9.8 directly so I don't have to symlink the DLL. I only use cygwin for Git and don't know the build system well enough to know how to link against old versions)