Convert linux sha512 shadow to hex

Question

I am trying to understand the linux shadow string format, and convert the string to a general hex format, just like how MD5 is usually represented.

I was following the definitions here http://www.akkadia.org/drepper/SHA-crypt.txt

As described, the final step (step 22) is a special base-64 encoding, with re-ordered bytes.

I wrote a python script(my first python program) to decode the base64 then put it back to the original order.

But THE PROBLEM IS, the result is not the same string as usual SHA1 hashes.

For example, a password "123" (without quotes) with salt "456" will produce a shadow string

$6$456$yTSeWYNbvZDCsuZIN.Qdeg.0DxY5N1XddpO7qgFqjnZOqpy5QXIeMM7pdQYWIgu6Y3pSh5eYqJ21fqrlrjhJe/

With my program, it generates

A922F952190B1ED9ADD9EFEDA918472364A10CABDBE79D7B5EA52A4FA6691B6A7648D429AB7BED45C7F7FE9938B8C0084F3025365C1FDC968A145192767D566A

However the SHA512 hash of 123456 and 456123 are

BA3253876AED6BC22D4A6FF53D8406C6AD864195ED144AB5C87621B6C233B548BAEAE6956DF346EC8C17F5EA10F35EE3CBC514797ED7DDD3145464E2A0BAB413

CA3D1DDE02C4B15D2E95521E259C5E08AAEA8FEAA722BA14014605249EFE3F248DB3D98AA7C4ACCBE887E1B40573D7EBA71017C5DF029C16C8D6F06B0FFDA310

None of them is the same as my result. So, is there anything I understood wrong about the shadow generation method? Or something wrong with my code?

my python code (input should be the base64 code, i.e. the string after the last "$" of a shadow string):

#!/usr/bin/python
# filename: conv.py
import sys
b64s = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
bm = ( \
        (0,21,42), \
        (22,43,1), \
        (44,2,23), \
        (3,24,45), \
        (25,46,4), \
        (47,5,26), \
        (6,27,48), \
        (28,49,7), \
        (50,8,29), \
        (9,30,51), \
        (31,52,10), \
        (53,11,32), \
        (12,33,54), \
        (34,55,13), \
        (56,14,35), \
        (15,36,57), \
        (37,58,16), \
        (59,17,38), \
        (18,39,60), \
        (40,61,19), \
        (62,20,41), \
        ('*','*',63))

sd_str = raw_input('Enter a Shadow String: ')
ba = [0]*64
ba_org = [0]*64
for i in range (0,21):
        byte24 = 0
        for j in range(0,4):
                byte24 += (b64s.find(sd_str[i*4+j]) << (j*6))

        for j in range(0,3):
                ba[bm[i][2-j]] = ((byte24>>(j*8))&0x0000FF)

byte24 = b64s.find(sd_str[84]) + (b64s.find(sd_str[85])<<6)
ba[63] = byte24        #last byte
for i in range (0,64):
        sys.stdout.write('%02X' % ba[i])

sys.stdout.write('\n')
sys.stdout.flush()

Answer 1

The biggest thing you're missing is that the password hash in the shadow file is iterated. The hash value for the password 123 is not sha(password + salt), it's sha(sha(sha(sha(password + salt) + salt) + salt)...), with 5000 levels of nesting by default.

Answer 2

The "sha256_crypt" algorithm is not just an SHA256 hash - it's rather complicated construction all it's own, which happens to use SHA256 as a cryptographic primitive, hence the name.

Not only does the rounds value require multiple compositions of the SHA256 function, it's not even sha256(sha256(...)), but a much more complicated loop, with a number of additional setup steps beforehand... the goal of which is to make it much more suitable for password hashing than simply sha256(). That said, the transposition before encoding to base64, and the custom base64 encoding, doesn't really add anything security-wise, it just makes more of a pain to implement.

I'd suggest reading the rest of the Akkdia.org document, which describes exactly how the shadow hash is calculated. Alternately, you can look at sha2_crypt.py in the source of the Passlib project, which shows what you need to implement - look for the raw_sha256_crypt() function, that's the right starting point, though the meat is in raw_sha_crypt(). Note that I linked you to the v1.5.3 source, because the latest version (while faster) is also a bit harder to understand.