AES 128 bit CTR partial file decryption with PHP

Question

This is not a duplicate post because I have looked everywhere and I can't find an answer to this. Its about partial decryption. Not full.

I have a good knowledge of PHP but little knowledge about cryptography. I know the key and the iv of the crypted file. The file is decrypting fine as whole but the real issue arises when I try to decrypt the partial file from the middle.

It decrypts fine when I try to decrypt the first 128kb of the file or 256kb or any length from the beginning of the file. but when I start from the middle, it does not decrypt but gives gibberish output.

I will post here an example of first 100 bytes of the file.

The encryption is AES 128 bit CTR mode.

I have used both mdecrypt_generic and mcrypt_decrypt functions of PHP with no success.

code used:

$chunk2 = mcrypt_decrypt(MCRYPT_RIJNDAEL_128, base64_decode($mega_key), $data, 'ctr', base64_decode($mega_iv));
echo base64_encode($data) .'<br />'. base64_encode($chunk2);

Result:

9PX2fU83NF3hLc+HFdyHkqfxC4bHWKUQwQHJkNVnYbKCIQrhlHvTKtz8T3Bb0TgBkyBoGHnDCzZs3bu54KLQ8Bv0lzrTVJbzJY5msBfcy7Zi2Z/fLoMm+nvqdGPTNR0uwv45xJ8=
MTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTE=

As you can see. after decryption, the result is the first 100 bytes of the file containing the digit 1 in series. The file is encrypted by Mega.co.nz using JavaScript. According to the documentation at Mega, the file has been encrypted in chunks for partial encryption/decryption.

File encryption

MEGA uses client-side encryption/decryption to end-to-end-protect file transfers and storage. Data received from clients is stored and transmitted verbatim; servers neither decrypt, nor re-encrypt, nor verify the encryption of incoming user files. All cryptographic processing is under the control of the end user. To allow for integrity-checked partial reads, a file is treated as a series of chunks. To simplify server-side processing, partial uploads can only start and end on a chunk boundary. Furthermore, partial downloads can only be integrity-checked if they fulfil the same criterion. Chunk boundaries are located at the following positions: 0 / 128K / 384K / 768K / 1280K / 1920K / 2688K / 3584K / 4608K /. (every 1024 KB) / EOF

I am calculating the chunk boundaries of the file with this function:

public function get_chunks($size)
{
    $chunks = array();
    $p = $pp = 0;

    for ($i = 1; $i <= 8 && $p < $size - $i * 0x20000; $i++) {
        $chunks[$p] = $i * 0x20000;
        $pp = $p;
        $p += $chunks[$p];
    }

    while ($p < $size) {
        $chunks[$p] = 0x100000;
        $pp = $p;
        $p += $chunks[$p];
    }

    $chunks[$pp] = ($size - $pp);
    if (!$chunks[$pp])
    {
        unset($chunks[$pp]);
    }

    return $chunks;
}

I have tried decrypting the 2nd chunk individually, it fails. I cannot post a 128kb chunk here for obvious reasons, but I will show you chunk starting from 2nd byte to the 100th byte. This is the result with the same code:

code used:

$chunk2 = mcrypt_decrypt(MCRYPT_RIJNDAEL_128, base64_decode($mega_key), $data, 'ctr', base64_decode($mega_iv));
echo base64_encode($data) .'<br />'. base64_encode($chunk2);

Result:

9fZ9Tzc0XeEtz4cV3IeSp/ELhsdYpRDBAcmQ1WdhsoIhCuGUe9Mq3PxPcFvROAGTIGgYecMLNmzdu7ngotDwG/SXOtNUlvMljmawF9zLtmLZn98ugyb6e+p0Y9M1HS7C/jnEnw==
MDK6A0kyWI3903mj+GokBGfLvHCuzITg8flodIM34gGSGtpE3pnIxxGCDhq72AijgnlBUIv5DGuAVzNoc0MR2t5SnNi281TnmtnnlvomTOWKd3HAnJTtsKCvJoHXGQLdDfbMag==

The results are the same with mcrypt_module_open('rijndael-128', '', 'ctr', '');

I need to partially decrypt a file because I am trying to code an open source download manager for Mega which supports parallel connections/resume support.

I need to decrypt the file on the fly to allow file streaming, so decrypting after its downloaded is out of question.

At mega's website, their own download interface uses multiple connections and downloads the file in chunks and there is another web service that allows downloading from Mega with multiple connections and resume support.

I need to decrypt the partial file to support HTTP Range header requests from the browser/download managers. If the range request falls with in the 2nd or 3rd block I need to be able to decrypt that block and send it to the client without decrypting the file from the beginning.

Is it even possible? It should be because some website has already done it.

Answer 1

Okay, I found the solution about decrypting partial content on the fly. I had this problem when I tried to decrypt partial data from Range ( in order to create a web proxy ). I discovered that if I download whole the file (from position 0 so), there is no problem. Example :

while($_total_dled != $content_length) {
        $raw = fgets($socket, 1024);
        $data = mdecrypt_generic($this->cipher, $raw);
        echo $data;
}

BUT, if I start reading the file from pos x ( like if I received : Range x-) , mdecrypt_generic will failed, because it wasn't be executed (x-1) times before. So I try this :

for($i=0;$i<$range["start"];$i++) {
     mdecrypt_generic($this->cipher, "0");
}

Just need to decrypt one character (x-1) times. and then, the part with the while loop. And it works. May be we can link this iteration with the related counter.

Hope this will help people.

Answer 2

Seems to me that you are doing a lot of calculations on the chunk boundaries (which are given as a table + simple calculation over 4608Ki) and no calculation to get to a new IV.

The IV is just the random nonce plus a block counter. So to calculate a new "IV" for a specific chunk you would have to do:

1. create the initial counter by shifting the nonce in the higher (left) bytes
2. get the chunk lower boundary
3. divide the lower boundary by the block size
4. add or XOR (the same for the first 2^64 blocks) the resulting number to the initial counter
5. initialize your cipher with it

Then you should be able to decrypt.