HTTP Proxy server [Python]

Question

I have this code here for an HTTP Proxy server which works. How do I create another program called "Client"? The client will send HTTP GET requests to multiple web servers via the proxy server. The client program connects to the proxy and sends HTTP GET requests for the following 3 websites: (www.google.com, www.yahoo.com, www.stackoverflow.com) with an interval of 30 seconds.

-My overall question is how do i send HTTP GET requests to the proxy server from python, not my web browser?

OSX 10.10.3 Python 3.4

When i call this proxy in my terminal:

python 1869.py 2000

You can give any port number in place of 2000.

Output:

starting server ....
Initiating server... 
 Accepting connection

Then in my browser (im using the most update version of chrome) I type in:

localhost:2000/www.stackoverflow.com

And my terminal output is:

request is  GET  to URL :  www.stackoverflow.com
/www.stackoverflow.com
File Present in Cache

HTTP/1.1 301 Moved Permanently

Content-Type: text/html; charset=UTF-8

Location: http://stackoverflow.com/

Date: Thu, 07 May 2015 17:45:40 GMT

Content-Length: 148

Connection: close

Age: 0



<head><title>Document Moved</title></head>

<body><h1>Object Moved</h1>This document may be found <a HREF="http://stackoverflow.com/">here</a></body>
Reading file from cache

Initiating server... 
 Accepting connection

Proxy code:

import socket
import sys
if len(sys.argv) <= 1: 
    print 'Usage: "python S.py port"\n[port : It is the port of the Proxy Server'
    sys.exit(2)

# Server socket created, bound and starting to listen
Serv_Port = int(sys.argv[1]) # sys.argv[1] is the port number entered by the user
Serv_Sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # socket.socket function creates a socket.

# Prepare a server socket
print "starting server ...."
Serv_Sock.bind(('', Serv_Port))
Serv_Sock.listen(5)


def caching_object(splitMessage, Cli_Sock):
    #this method is responsible for caching
    Req_Type = splitMessage[0]
    Req_path = splitMessage[1]
    Req_path = Req_path[1:]
    print "Request is ", Req_Type, " to URL : ", Req_path

    #Searching available cache if file exists
    file_to_use = "/" + Req_path
    print file_to_use
    try:
        file = open(file_to_use[1:], "r")
        data = file.readlines()
        print "File Present in Cache\n"

        #Proxy Server Will Send A Response Message
        #Cli_Sock.send("HTTP/1.0 200 OK\r\n")
        #Cli_Sock.send("Content-Type:text/html")
        #Cli_Sock.send("\r\n")

        #Proxy Server Will Send Data
        for i in range(0, len(data)):
            print (data[i])
            Cli_Sock.send(data[i])
        print "Reading file from cache\n"

    except IOError:
        print "File Doesn't Exists In Cache\n fetching file from server \n creating cache"
        serv_proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        host_name = Req_path
        print "HOST NAME:", host_name
        try:
            serv_proxy.connect((host_name, 80))
            print 'Socket connected to port 80 of the host'
            fileobj = serv_proxy.makefile('r', 0)
            fileobj.write("GET " + "http://" + Req_path + " HTTP/1.0\n\n")

            # Read the response into buffer
            buffer = fileobj.readlines()

            # Create a new file in the cache for the requested file.
            # Also send the response in the buffer to client socket
            # and the corresponding file in the cache
            tmpFile = open("./" + Req_path, "wb")
            for i in range(0, len(buffer)):
                tmpFile.write(buffer[i])
                Cli_Sock.send(buffer[i])
        except:
            print 'Illegal Request'

    Cli_Sock.close()
while True:
    # Start receiving data from the client
    print 'Initiating server... \n Accepting connection\n'
    Cli_Sock, addr = Serv_Sock.accept() # Accept a connection from client
    #print addr

    print ' connection received from: ', addr
    message = Cli_Sock.recv(1024) #Recieves data from Socket

    splitMessage = message.split()
    if len(splitMessage) <= 1:
        continue

    caching_object(splitMessage, Cli_Sock)

I don't see a question here (or even the description of a problem). — larsks
My overall question is how do i send HTTP GET requests to the proxy server from python, not my web browser? — Matt Montalbano

perevertysh perevertysh · Accepted Answer · 2019-02-06T18:22:14

you can use httpie from linux' terminal:

http [get/post] http://host:port/link_name/

in your app you can use requests: pip install requests

import requests

response = requests.get(url='url', proxies='proxies')
response.close()
print(response)

HTTP Proxy server [Python]

1 Answers