Git fetch fails due to package failure

Question

When I add our remote repository as up and try to extract it, it fails, as shown below:

$ git fetch upstream remote: Counting objects: 11901, done. remote: aborting due to possible repository corruption on the remote side. error: pack-objects died of signal 9 error: git upload-pack: git-pack-objects died with error. fatal: git upload-pack: aborting due to possible repository corruption on the re mote side. fatal: protocol error: bad pack header 

I understand that it fails due to the presence of huge files in the repository (which we have), but why it doesn’t work when I cloned the same repository? Because I can successfully clone the repository. Should the same objects be packed during the cloning request?

Answer 1

Expand Bit on VonC Response ...

Firstly, this may mean that signal 9 belongs to SIGKILL and tends to occur because the remote computer in question is the Linux host and the process is destroyed by the Linux "OOM killer" (although some non-Linux systems behave the same).

Next, talk about objects and packages. The git object "is one of four types of elements that are in the git repository:" blob "(file);" tree "(list of blobs, their modes and their name-as-stored-in-directory): that is, what will become a directory or folder when unpacking a commit);" commit "(which gives the author of the message, message and top-level tree among other data); and" tag "(annotated tag). Objects can be stored as" free objects ", and one object in a file by itself; but they can take a lot disk space, so instead of them can be "packed", many objects in one file with the addition of additional compression.

Creating a package from many loose objects that perform this compression is (or at least can be) a cpu and memory operation. The amount of memory required depends on the number of objects and their basic sizes: large files take up more memory. Many large files take up a lot of memory.

Further, as VonC noted, git clone skips trying to use thin packages (well, as a rule, anyway). This means that the server simply delivers batch files that already exist. This is a "cheap memory" operation: files already exist, and the server only needs their delivery.

On the other hand, git fetch tries, if possible, to avoid sending a lot of data that the client already has. Using a smart protocol, the client and server are involved in some kind of conversation that you can think of as something like this:

• "I have an object A that needs B and C, do you have B and C? I also have D, E and F."
• "I have B, but I need C, and I have D and E, please send me A, C and F."

Thus, the informed server extracts the "interesting" / "necessary" objects from the source packages, and then tries to compress them into a new (but "thin") package. This means that the server will call git-pack-objects .

If the server is low on memory (the “low” will refer to the amount git-pack-objects will need), it is likely to cause the “OOM killer”. Since git-pack-objects is memory intensive, this process is a likely candidate for killing the OOM killer. Then at your end you will see a message about git-pack-objects dying from signal 9 ( SIGKILL ).

(Of course, it is possible that the killer of the OOM server kills something completely different, for example, the error database server. :-))

Answer 2

This may be protocol dependent, but Documentation/technical/pack-heuristics.txt points to the first difference between clone and fetch.

In the other direction, fetch, git-fetch-pack and git-clone-pack call git-upload-pack on the other end (via ssh or talking to the daemon).

There are two cases:

• clone-pack and fetch-pack with -k save the downloaded batch file without extension, so we do not use thin packet transfer.
• Otherwise, the generated package will have a delta without a base object in the same package.

But fetch-pack without -k will explode the resulting package into separate objects, so we will automatically ask upload-pack to give us a thin package if upload-pack supports it.

Thus, in terms of protocols, Documentation/technical/pack-protocol.txt illustrates that fetching can return much more data than the git clone.