Incorrect decompression of Java Gzip in .NET

Question

I'm trying to repeat in .NET the algorithm that was originally written in Java and I'm having troubles with the GZIP decompression.

At the bottom of the post I inserted the hex string that is converted to byte array in both .NET and Java. The resulting byte array is then decompressed in Java with the following method:

public static Object readObjectFromByte(byte[] bytes)
{
ObjectInputStream oos = null;
try {
  ByteArrayInputStream baos = new ByteArrayInputStream(bytes);
  zis = new GZIPInputStream(baos);
  oos = new ObjectInputStream(zis);
  return oos.readObject();
} catch (Throwable t) { GZIPInputStream zis;
  return null;
} finally {
  try {
    if (oos != null) {
      oos.close();
    }
  } catch (IOException e) {
    e.printStackTrace();
  }
}
}

After decompression the resulting byte array has a length of 3952 which is probably correct. At the same time I tried different .NET classes/libs to decompress, but it always gives a byte array of 3979 bytes which is probably incorrect. I tried:

I read a lot of articles about GZIP issues in .NET trying to fix this. I use .NET 4.5, and for example my last decompression version is this:

Ionic.Zlib.GZipStream.UncompressBuffer(compressedBytes)

It's weird but even if I try:

Ionic.Zlib.GZipStream.CompressBuffer(Ionic.Zlib.GZipStream.UncompressBuffer(compressedBytes)).SequenceEquals(compressedBytes)

It gives me FALSE.

The hex string:

EDIT:

Java Code:

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;

public class JavaFiddle
{
public static void main(String[] args)
{
  String hex = "PLEASE_UPDATE"; //update this from the hex constant at the end of the post
  byte[] compressedBytes = hexStringToByteArray(hex);
  byte[] decompressedBytes = (byte[])readObjectFromByte(compressedBytes);
  System.out.println(decompressedBytes.length); //THIS GIVES 3952
}

public static byte[] hexStringToByteArray(String s) {
    int len = s.length();
    byte[] data = new byte[len / 2];
    for (int i = 0; i < len; i += 2) {
        data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                             + Character.digit(s.charAt(i+1), 16));
    }
    return data;
}

public static Object readObjectFromByte(byte[] bytes)
{
    ObjectInputStream oos = null;
    try {
      ByteArrayInputStream baos = new ByteArrayInputStream(bytes);
      GZIPInputStream zis = new GZIPInputStream(baos);
      oos = new ObjectInputStream(zis);
      return oos.readObject();
    } catch (Throwable t) { GZIPInputStream zis;
      return null;
    } finally {
      try {
        if (oos != null) {
          oos.close();
        }
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
}    
}

.NET Code

    private byte[] StringToByteArray(string hex)
    {
        int NumberChars = hex.Length;
        byte[] bytes = new byte[NumberChars / 2];
        for (int i = 0; i < NumberChars; i += 2)
            bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
        return bytes;
    }
    ...
    var hex = "PLEASE_UPDATE"; //update this from the hex constant at the end of the post
    var compressedBytes = StringToByteArray(hex);
    var decompressedBytes = Ionic.Zlib.GZipStream.UncompressBuffer(compressedBytes); 
    //decompressedBytes.Length is 3979, Note that this is using one of the external libraries, the same result is for built-in GZipStream in .NET

1F8B0800000000000000ADCEF923130A0000E0596C8A1CB90AA1D0E428916C4AE6889C61071E1D2A77B42C67D1731BF2DE6C662C5744C6DE7BC8DC4791DCB79E35315EC2C62437E1FDBE9FF7FD055FC53240282418007437AB5893DB060923594000201C030088639882D19C31C860FB7838A60DBBF039C3B41821FF606044C89D24EB4C1D6A627FBF8E978D3616796C2ACDD165ABB32B81A0C493A35F163889E74665A9341C7C36F06FA0F991201BEAAB6A47842C43A50DA6B8C000AD794840CF03E29A527D0C921FAE6D607C71AEF7C01DB34037A7F8CDD8CB1F61DB9C15A87DBBDD1F933FDF58312B054D7B5AF72FE75A0AA101714EEA6980BF405A15A1F13A0979FB5BBE1636A367CB4C943FB7F87A3245573BB6857DA6AA0B412CCB553B39E06973EB2A3864E98915E46C16E4EE06E9DA63DB642915AF6E86811FD2C8F3A1F325C82677EA701D1E3574885696DEB3B8A0C29AE1FDD41DA26C55C6381506A3CF77DD15BD37783F545F76E8DCF9FB6A8B0776E2427FBFF431B73748CE96786EA569E307B5DC8990A68A52838D9DC1BA19291F283BF2C7EE43D3E4516D77A07FDE04934BB7C254057BDFC3AD153A0A711E7EFDDDED31AED367FAAEA6F494A55E029BF6AECA9433CA7D45B52EC24BCF9074315BFE5F5494F4F0DDDE8D73F56397A0667992950E62AD4020A4102DBE7C4D6B85E457F184DC5AA51D2D536313E1E16636928BD6500207571B83C8830DB9E7CF2C92AE4DDB12B8DF3C6AAF83B09CAACB78AA78CDB0DB393D71A599F185B413BFDB907FC99598D2E56ADE701EE72DB71C9650E820DB452F7DD4904E4679FFD0CB6FEEDF7527AF21BE5A5404B8BAE4E9951AE5C88BF9B5D5FFC6447989445C7A2DB80B3E4A2DBCB51978BA1729F97BDCAD174D345417B96941BF2CC5C10539C952B2526770BA4637A5820DCF150E16AED3159AD9724A8DFCFEA8BADA2A08BD1A0C9B05B019CE61E01EDECF7F028B466E4B53A5F1FAE56CB137EF269E1946896D1A3DDCDB756364A96D8E47F3FB33E9B70BBED37FA8719F2D2EA89BBFB0CAFB71CC56947F7907439E9FE97C61D2A326F830592509674860BA7CC7E713EFD69D2724DED740A2959846CFE801B313BB1DAB45594D70E888F2A3063C67085BFDB345D64D116372F5BE42BB08ABBC6D65D90E1E50C849EDD9A6F977751C4DB5A454C66B997D6A0CA148827E03AEE30F1697392124A8C44C6A914C3C69C21447F9C5660665C5D0D3F758D57086D0938608E9BECE9BA95D359C41E255B0524C55EDC97816A76E32B29BF55BAE0EC1F71DBF3FF548DC2C181755D85ABCAFEDF23C248EF7D3C1580AB88D28CDCF9935F1DA797145A7F0B18603C1DA39EA8E969A2279F20F077E7F62BD5C1F7C095BF1B21D36561A254836F37EBC0AEB8C8EAE8EC725044EE47F107DE413DE804EED41473E4D3BEC93E9234255F9FDD1FD91F00B32BF13B2629C4373888813E5FD08F66AB7295D281E180ACE99A77952DE6DCD675B20E021A71E91F78B899A1FC2F8FD913C0E4D980B1D6CF13F7A76E823BDF75FDE8FC513C3C491F51305F1A483B81A11884254CB2DB396F864B08DDEDD005A90EF457E7F8EDFB64FF7C7443C6E2135183619ED69F27E1464BF862FB14A66509C379818517F6B2423D0D1BE794E4D9C7130D9F54AB19DDF9FC825CF8B3B5EBD8090811D372C0315CEFBB926701AC2B84CB4820D25DD2D71F2F6630FB3FA88A3EAAF9E44C709EFACC05FF2FBD342E844AB8A22F433B7C2161FEDE70BF07EF29A6B6355DAF2D33E7A2679C8A38511939EEE73DABDE98B021A9F0E21A5CFFEE6F7A7C44CE2358158F5B15F837CE96B75622FEFE702653D87222B9319D46D321EEDA628713D1B355A30E0D07A655F12A55B6397CFEFCFA3DB3B76DB80347FDCA613FD1F3F6022EF276A1666DDA3BE518CB634F67B185BD2016DB2DC20E56C1C9B58CD411710E802FCFED0E3E1E182C37A9684A007EF0DEBBBCB783F37A0D39E2B6FAB147A3BCD75B1C24FA5939BDD9501D5E0365A60E58BE429C821BF3F774DC4603D43CE355412CC3DBF47639DF7B33CFEF4138133255770F2659E44F2FC9EE285FCDCEC414B473CCB31BA39BA798CDF1FF28A3EBD7549341C8E809E8AABF836C7FB59DB4025ED1DC5BF1874EDF4511ECCC397050C8C3F404FB04BF52DF4F35411EBFCFEFCB70C2AF6EE8AFC32A539F624F0E30D47DE8F9138B0C9B24F03F5CD53E05CA77D63CAA8800CA567E4D60D8D4ECBA9F9FE3C75FADED32267AB07ED635AD14CB6A87B1F8D5B5BD43F7C04E14DB30935A6B6CA780F2FBD10CD2EAE8DD13D6C9FA89334A8ADEAC40D63C54B6E881F9F382D3056BF7DACFFC6914A6DF60F64E7C5FE8EB222FD71632318CDC5E3CF8EC48602D238E95FC06C9BDBBD3D85535DED236D1FFA4EB1F3939B334A222BBF7FE034EB1599A7E20DFF89F92E9B3EE0D45FA7103A9B8BE6F727EC6D338C464AAA1B6B8AF426BA2153793FF7C43EEE7A43E6038296B83E130BAA9D467DB7A9B8E29E18B8F4E4493DB3E1BE4CDC1ABA3074A8D69058420FB2F56BCCD3CAB0806FA9C1E825D6F18644CD9137AB9195DD63A8BD3BA9F5C9BF0AEC9F5B6E605B2C02CAA93FB10A5F03ADD7370A981F3A99F0F3CC2DA93C298292E9859C01BBB088FE6CD979FC5A02127305D63C72B319EDD48A0EE6BE431C17B3F8E4FFDAC1F5D9A9F4EB7443EE2D3CAD6A3508A918C48DBB7F89081B7087C94A60F9FD297839305DA999BE163F234E3ED7D8D8CDFBA16AE5327F3E9D5925DF533EF5FEA2F26D3784CFB578877E15799DB73DAEC8D767F9FD81597235251D2DC413C2E1F58887AF75783F8248EC923F57F93D81A88D9E3E1E045997FD3C40B1BF83CEFC9CC2CCC079197928232333623BE67DA6A70B4818095DEEC23ABEC73E4DC9E1A74E04AEC1D420EB533CF3C06C242E1612F85A560A2861A0E54C218AC1834FCCDBD2F61FBC3B32983F25B29F5967A7219C74A10C769002C1869E9DC926A99C9D1AC92CC8BC3A5D151D405693BA0DBE292D94F40B918AFCEE6851DFC215B3738975F1EFAA32923037D96656C2D2B94B015CD1DA5F43D0A8FFA063099BA0BFB246724C8E61150FB2126EDE245963F4454EBCFF5105BDA267D598E2D1E9985136AF4348D8FCF96F355B27AE6E6F5378415D8A8D1CAD3EC3426969BF5790DBA0C01EF9B7E86095D9313F74309E01C7BA00B9B0E85DA89900D6F89DEA5BCA9273C017B56139F9CF13E7DB74724240262CBDFCEC05FB5DDAA64CB98257C9AAF7AA48AAD36577B3D6F11CD0A074D5B15860B2791E8E8B87AB2A5B2419F61561E6ECF56746E2185EBFFF0F96F821B38B0F0000

Thanks,

Rather than going for something that is "probably correct", it would help if you'd start off with known initial data, compress it, then decompress it in both Java and .NET. It doesn't help that you haven't shown any of the attempts in .NET. Ideally, provide a minimal reproducible example - are you able to demonstrate this with a very small piece of data, that could be easily hard-coded? — Jon Skeet
Thanks, unfortunately I don't have control over Java source code, I only saw how this produced a different result and I was given that only HEX string. I will try to play with it and will update the post later. As for .NET example, I mentioned about my last decompression try: Ionic.Zlib.GZipStream.UncompressBuffer(compressedBytes) I will also update my post to include the code how I convert HEX string to byte array. — Ihor Deyneka
It's not particularly odd that if you use one gzip library to decompress and recompress, you get different results, btw. There are various tweaks that could change the compression results. — Jon Skeet
I have done several interop projects between .NET and Java systems, and there are often byte order mark differences... I'd start with that if I were in your situation. — Brian Driscoll

Jon Skeet Jon Skeet · Accepted Answer · 2017-09-19T12:18:45

Now we've got more of the Java code, we can see the problem: you've got an extra layer of serialization around your real data. That has nothing to do with compression really.

Here's an example to show what I mean:

import java.io.*;

public class Test {
    public static void main(String[] args) throws Exception {
        try (ByteArrayOutputStream output = new ByteArrayOutputStream()) {
            try (ObjectOutputStream oos = new ObjectOutputStream(output)) {
                oos.writeObject(new byte[5]);
            }
            byte[] data = output.toByteArray();
            System.out.println(data.length);
        }
    }
}

That's writing a byte array that's 10 bytes long - but the result is 32 bytes long, because of the extra "wrapper" information. Note that the extra 27 bytes is the same as the discrepancy you've seen.

Fundamentally, it's odd to wrap a byte array in this way, and if you can possibly change the original code, that would be for the best. If you absolutely can't do that, it may be safe to just ignore the first 27 bytes of the resulting data.

Incorrect decompression of Java Gzip in .NET

1 Answers