Forum Discussion

cblown's avatar
cblown
Boss
7 years ago

Problems with Freemarker - utils.digest.hmac Hmac SHA256 and base 64 encoding?

Hi  @GavinD @ChhamaJ @YuriK DougS

 

We are trying to use http.client to access a 3rd party system. The API requires us to hash (using HMAC SHA 256) a challenge string with our secret key to gain an access token.

 

1. <#assign signature = utils.digest.hmac("HmacSHA256", "12345678901234567890", "the quick brown fox jumps over the lazy dog") />

Result = XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4=

 

Unfortunately we get “Unauthorised” from the remote server. So we tried a simple quick test in PHP

 

2. echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", "12345678901234567890" , true));

Result = XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4=

 

Ok PHP and Freemarker hash routines agree with the same arguments. Great so what gives? We did some digging and discovered via some sample client code that they base 64 decoded the key. So sticking with PHP we tried

 

4. echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", base64_decode("12345678901234567890") , true));

Result = TWxBl/zreSUOx/NFx+aHNLTU9tPkQ7TLQDPngthWf20=

And this worked !! We now got an access token from the remote service.. So we thought awesome lets do the same in Freemarker

 

5. <#assign signature = utils.digest.hmac("HmacSHA256", utils.base64.decode("12345678901234567890”), "the quick brown fox jumps over the lazy dog") />

Result = Y+4vRRdPkXF5uRWjtIkEJosBpCHVEb0AqXRPfaNl5Tc=

 

But we get a different hash? and still no token :(

 

We suspect something weird is going on with string casting arguments in the Freemarker - because if we try this

 

utils.base64.encode( utils.base64.decode("12345678901234567890”))  

We get this 77+9be+/ve+/ve+/ve+/vU12345677+977+9dA== 

 

With php echo base64_encode( base64_decode("12345678901234567890") ) 

as expected 12345678901234567890

 

Any ideas?

 

  • DougS's avatar
    DougS
    Khoros Oracle
    7 years ago

    Hi cblown

     

    I think I might know what's going on here, but please correct me if I've got something wrong:

     

    Base64 encoding was designed to encode binary data into an ASCII string (a radix-64 representation). The string "12345678901234567890" is not a base-64-encoded string, so decoding it using base64 will not necessarily generate the same result on different implementations of the Base64 spec (since the spec says nothing about this). Try any number of online Base64 decoding tools and try decoding the string "12345678901234567890" -- you will either get different results in different tools, or you may get an error (because this is not a valid use of Base64 decoding).

     

    If you encode the string "12345678901234567890" using base64, you will get the same results in the Lithium's Freemarker context implementation that you get in any other valid Base64 encoding algorithm (for UTF-8 charset you would get "MTIzNDU2Nzg5MDEyMzQ1Njc4OTA="). If you then decode the base-64 encoded string (in this case "MTIzNDU2Nzg5MDEyMzQ1Njc4OTA=") you will get the same result on  in Lithium's Freemarker context implementation that will get in any other valid Base64 decoding algorithm (for UTF-8 charset you would get "12345678901234567890").

     

    Why is the 3rd party auth system trying to decode the string "12345678901234567890" (which isn't a base-64 encoded string), and can that be changed? (because it's not going to behave correctly on different implementations of Base64).

     

    -Doug

    • cblown's avatar
      cblown
      Boss
      7 years ago

      DougS - thanks for the reply. The key I'm using above is just an example and was a bad example in hindsight :) - I'll PM'd you the real key.. I'm thinking now the key might be the problem and as you say each implementation is potentially different if the encode string is not 100% right. 

       

      I did some further testing using your example MTIzNDU2Nzg5MDEyMzQ1Njc4OTA= since we've confirmed its encoded. It works as expected. I suspect the key we received from the 3rd party site, as you mentioned, not properly encoded, although with php we can get a token using this key. 

       

      PHP

      echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", "12345678901234567890" , true));
echo "<br>";
echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", base64_decode("MTIzNDU2Nzg5MDEyMzQ1Njc4OTA=") , true));

      Returns

      XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4=
XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4=

      Freemarker

      <#assign signature = utils.digest.hmac("HmacSHA256", "12345678901234567890", "the quick brown fox jumps over the lazy dog") />
<#assign signature2 = utils.digest.hmac("HmacSHA256", utils.base64.decode("MTIzNDU2Nzg5MDEyMzQ1Njc4OTA="), "the quick brown fox jumps over the lazy dog") />
${signature}
<br>
${signature2}

      Returns

      XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4= 
XIDINszyYHIP5vszuWRnyP/4/p/y0COFZHV8tzcxPh4=

       

       

       

      • cblown's avatar
        cblown
        Boss
        7 years ago

        Actually, you know what I don't think the key is all that sensitive on its own, its a testing key anyway and you need an app Id and other stuff.. So here it is again with the real key :)

         

        PHP

        echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", base64_decode("2ZKF2RDckw3M++TwLVP8t+AQIU66Rj3mCykTbMwBnOWUpAnM") , true));

        Result

        2IweFAZF3mu13aizUcTjRV450W1LEwISUJnRnOapOUk=

          

        Freemarker

        <#assign signature = utils.digest.hmac("HmacSHA256", utils.base64.decode("2ZKF2RDckw3M++TwLVP8t+AQIU66Rj3mCykTbMwBnOWUpAnM"), "the quick brown fox jumps over the lazy dog") />
${signature}

        Result

        w868e7nxYVl+qNTNMg6Zb4rLOh02x0jC5i8LYfkeVuE=

         

  • RobertT's avatar
    RobertT
    Boss
    7 years ago

    cblown call me daft but are you not better simply storing the hash output you generated using PHP and setting the variable to that, rather than trying to generate this in FreeMarker?

     

    If the challenge and secret key are unlikely to change then it shouldn't cause any problems. This would also be a bit more secure as the challenge string and secret key are not stored then on the community only the hashed output.

     

    There is cleary something different in the way the hash is genered between PHP and FreeMarker, for example I notice that you wrap the entire code in base64_encode for PHP:

     

    echo base64_encode(hash_hmac('sha256', "the quick brown fox jumps over the lazy dog", base64_decode("12345678901234567890") , true));

    but don't do the same in FreeMarker, you're only decoding the key:

    <#assign signature = utils.digest.hmac("HmacSHA256", utils.base64.decode("12345678901234567890”), "the quick brown fox jumps over the lazy dog") />

    There could be something in that which we're missing.

     

     

    • cblown's avatar
      cblown
      Boss
      7 years ago

      The access token expires so we have to hash another challenge each time to get a new token. The Freemarker routine returns base 64 as per the docs so no need to encode it like we do for the php. The php and freemarker agree on the hash, it’s only when we try to pass a byte array to the freemarker (the output from the base 64 decode should be straight binary) The fact I can’t decode and re-encode in Base 64 using Freemarker is somehow related.. 

      • luk's avatar
        luk
        Boss
        7 years ago

        Nice problem, don't know the solution though =)...I would ping DougS for things like this!