python - "Clicking" button with requests

6
votes

I have this little website i want to fill in a form with the requests library. The problem is i cant get to the next site when filling the form data and hitting the button(Enter does not work).

The important thing is I can't do it via a clicking bot of some kind. This needs to be done so I can run in without graphics.

info = {'name':'JohnJohn',
        'message':'XXX',
        'sign':"XXX",
        'step':'1'}

First three entries name, message, sign are the text areas and step is I think the button.

r = requests.get(url)
r = requests.post(url, data=info)

print(r.text)

The Form Data looks like this when i send a request via chrome manually:

  • name:JohnJohn
  • message:XXX
  • sign:XXX
  • step:1

The button element looks like this:

<td colspan="2" style="text-align: center;">
    <input name="step" type="hidden" value="1">
    <button id="button" type="button" onclick="myClick();"
     style="background-color: #ef4023;  width: 80px; font-face: times; font-size: 14pt;">
        Wyślij
    </button>
</td>

The next site if i do this manually has the same adres.

2
pythonweb-scrapingpython-requestsbots
Can you share the link? - Padraic Cunningham

2 Answers

15
votes

As you might see from the snipped you posted, clicking the button is triggering some JavaScript code, namely a method called myClick().

It is not straightforward to click on this thing using pythons requests library. You might have more luck trying to find out what happens inside myClick(). My guess would be that at some point, a POST request will be made to a HTTP endpoint. If you can figure this out, you can translate it into your python code.

If that does not work, another option would be to use something like Selenium/PhantomJS, which give you the ability to have a real, headless and scriptable browser. Using such a tool, you can actually have it fill out forms and click buttons. You can have a look at this so answer, as it shows you how to use Selenium+PhantomJS from python.

Please make sure not to abuse such methods by spamming forums or [insert illegal or otherwise abusive activity here].

2
votes

In such a situation when you need to forge scripted button's request, it may be easier not to guess the logic of JS but instead perform a physical click and look into chrome devtools' network sniffer which gives you a plain request made which, in turn, can be easily forged in Python