HTTP request using cURL works but doesn't work in Python requests

Question

I am trying to fetch a HTTP request response using cURL, which works fine. However, when I use 'requests' package in Python I am not getting any response (it just times out, no error or response returned).

My cURL code as follows:

curl 'http://144.126.138.12:17001/api/v0/get-posts-stateless' \
  -H 'authority: bitclout.com' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"' \
  -H 'accept: application/json, text/plain, */*' \
  -H 'sec-ch-ua-mobile: ?1' \
  -H 'user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Mobile Safari/537.36' \
  -H 'content-type: application/json' \
  -H 'origin: https://bitclout.com' \
  -H 'sec-fetch-site: same-origin' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-dest: empty' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'cookie: datadome=2P0n.mJzUY2hB5B03Tc2hild-xu2AkuX~-mIgJ2VRW-j35yl3njRPJEKTAHgcxyQdxrX-E3vQnr5GSEA~DfK3nyrtPJB1M77Ndccn3BZnf; INGRESSCOOKIE=389b5e3fc60c40d6115364749da12fd0; amp_23345b=c6QQHrpE_pvlOyAXAEKiaE.QkMxWUxpRWIzMVVHWHZ4bUZ0N0piQktIS2pjRjJTMUhZNVNkOEMzWlp5ZTFmQnVEVTVOZnNoYw==..1f6htdgom.1f6htgqqm.4r.0.4r' \
  --data-raw '{"PostHashHex":"","ReaderPublicKeyBase58Check":"BC1YLiUgto51FaPyCPwf2LGGQcCo4qa3XM3KY1zAbK8jCzaDCc4bU27","OrderBy":"newest","StartTstampSecs":null,"PostContent":"","NumToFetch":400,"FetchSubcomments":false,"GetPostsForFollowFeed":false,"GetPostsForGlobalWhitelist":false,"GetPostsByClout":false,"PostsByCloutMinutesLookback":0,"AddGlobalFeedBool":false}' \
  --compressed

When I use -v with the cURL code then I get the following message:

Trying 144.126.138.12...

TCP_NODELAY set

Connected to 144.126.138.12 (144.126.138.12) port 17001 (#0)

POST /api/v0/get-posts-stateless HTTP/1.1 Host: 144.126.138.12:17001 Accept-Encoding: deflate, gzip authority: bitclout.com sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90" accept: application/json, text/plain, / sec-ch-ua-mobile: ?1 user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Mobile Safari/537.36 content-type: application/json origin: https://bitclout.com sec-fetch-site: same-origin sec-fetch-mode: cors sec-fetch-dest: empty accept-language: en-US,en;q=0.9 cookie: datadome=2P0n.mJzUY2hB5B03Tc2hild-xu2AkuX~-mIgJ2VRW-j35yl3njRPJEKTAHgcxyQdxrX-E3vQnr5GSEA~DfK3nyrtPJB1M77Ndccn3BZnf; INGRESSCOOKIE=389b5e3fc60c40d6115364749da12fd0; amp_23345b=c6QQHrpE_pvlOyAXAEKiaE.QkMxWUxpRWIzMVVHWHZ4bUZ0N0piQktIS2pjRjJTMUhZNVNkOEMzWlp5ZTFmQnVEVTVOZnNoYw==..1f6htdgom.1f6htgqqm.4r.0.4r Content-Length: 351

upload completely sent off: 351 out of 351 bytes < HTTP/1.1 200 OK < Access-Control-Allow-Credentials: true < Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept < Access-Control-Allow-Methods: GET, PUT, POST, DELETE, OPTIONS < Access-Control-Allow-Origin: https://bitclout.com < Content-Security-Policy: default-src 'self' < Content-Type: application/json < Referrer-Policy: same-origin < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Xss-Protection: 1; mode=block < Date: Thu, 27 May 2021 20:21:56 GMT < Transfer-Encoding: chunked

Connection #0 to host 144.126.138.12 left intact

Closing connection 0

My Python code using requests as follows:

import requests

headers = {
    'authority': 'bitclout.com',
    'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"',
    'accept': 'application/json, text/plain, */*',
    'sec-ch-ua-mobile': '?1',
    'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Mobile Safari/537.36',
    'content-type': 'application/json',
    'origin': 'https://bitclout.com',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    'accept-language': 'en-US,en;q=0.9',
    'cookie': 'datadome=2P0n.mJzUY2hB5B03Tc2hild-xu2AkuX~-mIgJ2VRW-j35yl3njRPJEKTAHgcxyQdxrX-E3vQnr5GSEA~DfK3nyrtPJB1M77Ndccn3BZnf; INGRESSCOOKIE=389b5e3fc60c40d6115364749da12fd0; amp_23345b=c6QQHrpE_pvlOyAXAEKiaE.QkMxWUxpRWIzMVVHWHZ4bUZ0N0piQktIS2pjRjJTMUhZNVNkOEMzWlp5ZTFmQnVEVTVOZnNoYw==..1f6htdgom.1f6htgqqm.4r.0.4r',
}

data = '{"PostHashHex":"","ReaderPublicKeyBase58Check":"BC1YLiUgto51FaPyCPwf2LGGQcCo4qa3XM3KY1zAbK8jCzaDCc4bU27","OrderBy":"newest","StartTstampSecs":null,"PostContent":"","NumToFetch":400,"FetchSubcomments":false,"GetPostsForFollowFeed":false,"GetPostsForGlobalWhitelist":false,"GetPostsByClout":false,"PostsByCloutMinutesLookback":0,"AddGlobalFeedBool":false}'

response = requests.post('http://144.126.138.12:17001/api/v0/get-posts-stateless', headers=headers, data=data)
print(response.text)

Why is this happening and how can I make my Python code work?

You didn't post the error message or info about the response? — Paul Becotte
@PaulBecotte There is no error. It just times out. I have also updated my question to reflect this as well. — Somdip Dey
times out and throws an exception? Or just doesn't print any output? If you add -v to your curl command... is it using a proxy? — Paul Becotte
@PaulBecotte it doesn't print anything. no error at all. Its not using a proxy (i guess). I have updated my answer with -v on cURL code as well. — Somdip Dey
its not supposed to print anything... you're capturing the response in a variable. Try print(response.text) — Paul Becotte

ozcnakd ozcnakd · Accepted Answer · 2021-05-27T20:44:25

Can you try data like this:

data = "{\"PostHashHex\":\"\",\"ReaderPublicKeyBase58Check\":\"BC1YLiUgto51FaPyCPwf2LGGQcCo4qa3XM3KY1zAbK8jCzaDCc4bU27\",\"OrderBy\":\"newest\",\"StartTstampSecs\":null,\"PostContent\":\"\",\"NumToFetch\":400,\"FetchSubcomments\":false,\"GetPostsForFollowFeed\":false,\"GetPostsForGlobalWhitelist\":false,\"GetPostsByClout\":false,\"PostsByCloutMinutesLookback\":0,\"AddGlobalFeedBool\":false}"

HTTP request using cURL works but doesn't work in Python requests

2 Answers