Error 405 when trying to Post a Request with Python

Question

When I try to to create a Post Request, i just get error 405 or get forwarded to the homepage. I'm trying to send headers and authentication. What do I need to change in order to be able to connect to the URL?

I already tried swapping the order of data=auth, and headers=headers when connecting but it didn't do anything, also I tried another Website that didn't use csrf-tokens, but it also failed.


import requests                                                                
from bs4 import BeautifulSoup                       


# need to capture a valid csrf token                                           
# first visit the login page to generate one                                   
s = requests.session()                                                         
response = s.get('https://www.klickaud.com/')                              

# extract the token                                                            
soup = BeautifulSoup(response.text)                                            
for n in soup('input'):                                                        
    if n['name'] == 'testdummy':                                             
        token = n['value']                                                     
        break  



tokencsrf ='testdummy =' + token    

# now post to that login page with some valid credentials and the token        
auth = {                                                                       
     'value': 'https://soundcloud.com/bxxmbastic/fygb-flip'                                                           
    ,'testdummy': token 

}

headers = {

    'cookie': '__cfduid=d6cd11b0c476cdcd9364e010aebc3e1b01555296698; PHPSESSID=2eh4q7fndr2srru232bbeqc036'        
    'origin: https://www.klickaud.com'
    ,'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36'


}                                                                              
s.post('https://www.klickaud.com/download.php',headers=headers,data=auth)                             

#now we should be authenticated, try visiting a protected page                
response = s.post('https://www.klickaud.com/download.php', headers=headers, data=auth)                              
print(response.text)

I want to be able to parse a website with beautifulsoup, but when i request it I either get Error 405 telling me the headers are incorrect or I get forwarded to the Homepage.

ip. ip. · Accepted Answer · 2019-04-15T13:29:51

You don't need pass cookies in headers, because session do it
You have wrong headers dict (check quotes and commas)
You don't need 'authenticate', do only one post request

Here's working example:

import requests
from bs4 import BeautifulSoup

headers = {
    'origin': 'https://www.klickaud.com',
    'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
}

# need to capture a valid csrf token                                           
# first visit the login page to generate one
s = requests.session()
response = s.get('https://www.klickaud.com/', headers=headers)

# extract the token
soup = BeautifulSoup(response.text)
for n in soup('input'):
    if n['name'] == 'testdummy':
        token = n['value']
        break


data = {
    'value': 'https://soundcloud.com/bxxmbastic/fygb-flip',
    'testdummy': token
}

# send post data
response = s.post('https://www.klickaud.com/download.php', headers=headers, data=data)
print(response.text)

Error 405 when trying to Post a Request with Python

1 Answers