fatal: early EOF fatal: index-pack failed

Question

I have googled and found many solutions but none work for me.

I am trying to clone from one machine by connecting to the remote server which is in the LAN network.
Running this command from another machine cause error.
But running the SAME clone command using git://192.168.8.5 ... at the server it's okay and successful.

Any ideas ?

user@USER ~
$ git clone  -v git://192.168.8.5/butterfly025.git
Cloning into 'butterfly025'...
remote: Counting objects: 4846, done.
remote: Compressing objects: 100% (3256/3256), done.
fatal: read error: Invalid argument, 255.05 MiB | 1.35 MiB/s
fatal: early EOF
fatal: index-pack failed

I have added this config in .gitconfig but no help also.
Using the git version 1.8.5.2.msysgit.0

[core]
    compression = -1

Answer 1

First, turn off compression:

git config --global core.compression 0

Next, let's do a partial clone to truncate the amount of info coming down:

git clone --depth 1 <repo_URI>

When that works, go into the new directory and retrieve the rest of the clone:

git fetch --unshallow 

or, alternately,

git fetch --depth=2147483647

Now, do a regular pull:

git pull --all

I think there is a glitch with msysgit in the 1.8.x versions that exacerbates these symptoms, so another option is to try with an earlier version of git (<= 1.8.3, I think).

Answer 2

This error may occur for memory needs of git. You can add these lines to your global git configuration file, which is .gitconfig in $USER_HOME, in order to fix that problem.

[core] 
packedGitLimit = 512m 
packedGitWindowSize = 512m 
[pack] 
deltaCacheSize = 2047m 
packSizeLimit = 2047m 
windowMemory = 2047m

Answer 3

finally solved by git config --global core.compression 9

From a BitBucket issue thread:

I tried almost five times, and it still happen.

Then I tried to use better compression and it worked!

git config --global core.compression 9

From the Git Documentation:

core.compression
An integer -1..9, indicating a default compression level. -1 is the zlib default.
0 means no compression, and 1..9 are various speed/size tradeoffs, 9 being slowest.
If set, this provides a default to other compression variables, such as core.looseCompression and pack.compression.

Answer 4

As @ingyhere said:

Shallow Clone

First, turn off compression:

git config --global core.compression 0

Next, let's do a partial clone to truncate the amount of info coming down:

git clone --depth 1 <repo_URI>

When that works, go into the new directory and retrieve the rest of the clone:

git fetch --unshallow

or, alternately,

git fetch --depth=2147483647

Now, do a pull:

git pull --all

---

Then to solve the problem of your local branch only tracking master

open your git config file (.git/config) in the editor of your choice

where it says:

[remote "origin"]
    url=<git repo url>
    fetch = +refs/heads/master:refs/remotes/origin/master

change the line

fetch = +refs/heads/master:refs/remotes/origin/master

to

fetch = +refs/heads/*:refs/remotes/origin/*

Do a git fetch and git will pull all your remote branches now

Answer 5

In my case this was quite helpful:

git clone --depth 1 --branch $BRANCH $URL

This will limit the checkout to mentioned branch only, hence will speed up the process.

Hope this will help.

Answer 6

I tried all of that commands and none works for me, but what works was change the git_url to http instead ssh

if is clone command do :

git clone <your_http_or_https_repo_url> 

else if you are pulling on existing repo, do it with

git remote set-url origin <your_http_or_https_repo_url>

hope this help someone!

Answer 7

I got this error when git ran out of memory.

Freeing up some memory (in this case: letting a compile job finish) and trying again worked for me.

Answer 8

In my case it was a connection problem. I was connected to an internal wifi network, in which I had limited access to ressources. That was letting git do the fetch but at a certain time it crashed. This means it can be a network-connection problem. Check if everything is running properly: Antivirus, Firewall, etc.

The answer of elin3t is therefore important because ssh improves the performance of the downloading so that network problems can be avoided

Answer 9

Setting below's config doesn't work for me.

[core] 
packedGitLimit = 512m 
packedGitWindowSize = 512m 
[pack] 
deltaCacheSize = 2047m 
packSizeLimit = 2047m 
windowMemory = 2047m

As previous comment, it might the memory issue from git. Thus, I try to reduce working threads(from 32 to 8). So that it won't get much data from server at the same time. Then I also add "-f " to force to sync other projects.

-f: Proceed with syncing other projects even if a project fails to sync.

Then it works fine now.

repo sync -f -j8

Answer 10

A previous answer recommends setting to 512m. I'd say there are reasons to think that's counterproductive on a 64bit architecture. The documentation for core.packedGitLimit says:

Default is 256 MiB on 32 bit platforms and 32 TiB (effectively unlimited) on 64 bit platforms. This should be reasonable for all users/operating systems, except on the largest projects. You probably do not need to adjust this value.

If you want to try it out check if you have it set and then remove the setting:

git config --show-origin core.packedGitLimit
git config --unset --global core.packedGitLimit

Answer 11

I was getting the same error, on my side i resolved by running this command, In windows it has some memory issue.

git config --global pack.windowsMemory 256m

Answer 12

In my case nothing worked when the protocol was https, then I switched to ssh, and ensured, I pulled the repo from last commit and not entire history, and also specific branch. This helped me:

git clone --depth 1 "ssh:.git" --branch “specific_branch”

Answer 13

Note that Git 2.13.x/2.14 (Q3 2017) does raise the default core.packedGitLimit which influences git fetch:
The default packed-git limit value has been raised on larger platforms (from 8 GiB to 32 GiB) to save "git fetch" from a (recoverable) failure while "gc" is running in parallel.

See commit be4ca29 (20 Apr 2017) by David Turner (csusbdt).
Helped-by: Jeff King (peff).
^{(Merged by Junio C Hamano -- gitster -- in commit d97141b, 16 May 2017)}

Increase core.packedGitLimit

When core.packedGitLimit is exceeded, git will close packs.
If there is a repack operation going on in parallel with a fetch, the fetch might open a pack, and then be forced to close it due to packedGitLimit being hit.
The repack could then delete the pack out from under the fetch, causing the fetch to fail.

Increase core.packedGitLimit's default value to prevent this.

On current 64-bit x86_64 machines, 48 bits of address space are available.
It appears that 64-bit ARM machines have no standard amount of address space (that is, it varies by manufacturer), and IA64 and POWER machines have the full 64 bits.
So 48 bits is the only limit that we can reasonably care about. We reserve a few bits of the 48-bit address space for the kernel's use (this is not strictly necessary, but it's better to be safe), and use up to the remaining 45.
No git repository will be anywhere near this large any time soon, so this should prevent the failure.

Answer 14

I have the same problem. Following the first step above i was able to clone, but I cannot do anything else. Can't fetch, pull or checkout old branches.

Each command runs much slower than usual, then dies after compressing the objects.

I:\dev [master +0 ~6 -0]> git fetch --unshallow
remote: Counting objects: 645483, done.
remote: Compressing objects: 100% (136865/136865), done.

error: RPC failed; result=18, HTTP code = 20082 MiB | 6.26 MiB/s

fatal: early EOF

fatal: The remote end hung up unexpectedly

fatal: index-pack failed

This also happens when your ref's are using too much memory. Pruning the memory fixed this for me. Just add a limit to what you fetching like so ->

git fetch --depth=100

This will fetch the files but with the last 100 edits in their histories. After this, you can do any command just fine and at normal speed.

Answer 15

Using @cmpickle answer, I built a script to simplify the clone process.

It is hosted here: https://gist.github.com/gianlucaparadise/10286e0b1c5409bd1049d67640fb7c03

You can run it using the following line:

curl -sL https://git.io/JvtZ5 | sh -s repo_uri repo_folder

Answer 16

Tangentially related and only useful in case you have no root access and manually extract Git from an RPM (with rpm2cpio) or other package (.deb, ..) into a subfolder. Typical use case: you try to use a newer version of Git over the outdated one on a corporate server.

If git clone fails with fatal: index-pack failed without early EOF mention but instead a help message about usage: git index-pack, there is a version mismatch and you need to run git with the --exec-path parameter:

git --exec-path=path/to/subfoldered/git/usr/bin/git clone <repo>

In order to have this happen automatically, specify in your ~/.bashrc:

export GIT_EXEC_PATH=path/to/subfoldered/git/usr/libexec

Answer 17

I had the same problem, I even tried to download the project directly from the website as a zip file but the download got interrupted at the exact same percent.

This single line fixed my problem like a charm

git config --global core.compression 0

I know other answers have mentioned this but, no one here mentioned that this line alone can fix the problem.

Hope it helps.

Answer 18

It's confusing because Git logs may suggest any connection or ssh authorization errors, eg: ssh_dispatch_run_fatal: Connection to x.x.x.x port yy: message authentication code incorrect, the remote end hung up unexpectedly, early EOF.

Server-side solution

Let's optimize git repository on the server side:

1. Enter to my server's git bare repository.
2. Call git gc.
3. Call git repack -A

Eg:

ssh admin@my_server_url.com
sudo su git
cd /home/git/my_repo_name # where my server's bare repository exists.
git gc
git repack -A

Now I am able clone this repository without errors, e.g. on the client side:

git clone git@my_server_url.com:my_repo_name

The command git gc may be called at the git client side to avoid similar git push problem.

---

If you are an administrator of Gitlab service - trigger Housekeeping manually. It calls internally git gc or git repack.

---

Client-side solution

Other (hack, client-side only) solution is downloading last master without history:

git clone --single-branch --depth=1 git@my_server_url.com:my_repo_name

There is a chance that buffer overflow will not occur.

Answer 19

In my case the problem was none of the git configuration parameters but the fact that my repository had one file exceeding the maximum file size allowed on my system. I was able to check it trying to download a large file and getting an "File Size Limit Exceeded" on Debian.

After that I edited my /etc/security/limits.conf file adding et the end of it the following lines:

• hard fsize 1000000
• soft fsize 1000000

To actually "apply" the new limit values you need to re-login

Answer 20

From a git clone, I was getting:

error: inflate: data stream error (unknown compression method)
fatal: serious inflate inconsistency
fatal: index-pack failed

After rebooting my machine, I was able to clone the repo fine.

Answer 21

I turned off all the downloads I was doing in the meantime, which freed some space probably and cleared up/down bandwidth

Answer 22

The git-daemon issue seems to have been resolved in v2.17.0 (verified with a non working v2.16.2.1). I.e. workaround of selecting text in console to "lock output buffer" should no longer be required.

From https://github.com/git/git/blob/v2.17.0/Documentation/RelNotes/2.17.0.txt:

• Assorted fixes to "git daemon". (merge ed15e58efe jk/daemon-fixes later to maint).

Answer 23

Tried most of the answers here, I got the error with the PUTTY SSH Client with all possible constellations.

Once I switched to OpenSSH the error was gone (remove the Environment Variable GIT_SSH and restart the git bash).

I was using a new machine and newest git versions. On many other/older machines (AWS as well) it did work as expected with PUTTY as well without any git configuration.

Answer 24

I've experience the same problem. The REPO was too big to be downloaded via SSH. Just like @elin3t recommended, I've cloned over HTTP/HTTPS and change the REMOTE URL in .git/config to use the SSH REPO.

Answer 25

I got the same issue as below when I run git pull

remote: Counting objects: 149, done.
Connection to git-codecommit.us-east-1.amazonaws.com closed by remote host.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

Then I checked the git status, There were so many uncommitted changes I fixed the issue by committing and push all the uncommitted changes.

Answer 26

None of the solutions above worked for me.

The solution that finally worked for me was switching SSH client. GIT_SSH environment variable was set to the OpenSSH provided by Windows Server 2019. Version 7.7.2.1

C:\Windows\System32\OpenSSH\ssh.exe

I simply installed putty, 0.72

choco install putty

And changed GIT_SSH to

C:\ProgramData\chocolatey\lib\putty.portable\tools\PLINK.EXE

Answer 27

I tried pretty much all the suggestions made here but none worked. For us the issue was temperamental and became worse and worse the larger the repos became (on our Jenkins Windows build slave).

It ended up being the version of ssh being used by git. Git was configured to use some version of Open SSH, specified in the users .gitconfig file via the core.sshCommand variable. Removing that line fixed it. I believe this is because Windows now ships with a more reliable / compatible version of SSH which gets used by default.

Answer 28

In my case, I simply upgraded my version of OpenSSL. The older versions of OpenSSL have vulnerabilities and also do not have the latest algorithms which may be needed. As of today, the command openssl version shows OpenSSL 1.1.1f 31 Mar 2020.

Answer 29

I have tried for several times after I set git buffer, as I mentioned in the question, it seems work now.

So if you met this error, run this command:

git config --global http.postBuffer 2M

and then try again for some times.

Reference:

git push error: RPC failed; result=56, HTTP code = 0

Answer 30

Network quality matters, try to switch to a different network. What helped me was changing my Internet connection from Virgin Media high speed land-based broadband to a hotspot on my phone.

Before that I tried the accepted answer to limit clone size, tried switching between 64 and 32 bit versions, tried disabling the git file cache, none of them helped.

Then I switched to the connection via my mobile, and the first step (git clone --depth 1 <repo_URI>) succeeded. Switched back to my broadband, but the next step (git fetch --unshallow) also failed. So I deleted the code cloned so far, switched to the mobile network tried again the default way (git clone <repo_URI>) and it succeeded without any issues.