tcp python socket hold connection forever

Question

I have a client-server model where the client will constantly checking a log file and as soon as a new line comes in the log file it sends that line to the server.

Somehow I managed to work this thing using the following code.

server.py

import SocketServer


class MyTCPSocketHandler(SocketServer.BaseRequestHandler):

    def handle(self):
        # self.request is the TCP socket connected to the client
        data = self.request.recv(1024).strip()
        print data
        # process the data..


if __name__ == "__main__":

    HOST, PORT = "localhost", 9999
    server = SocketServer.TCPServer((HOST, PORT), MyTCPSocketHandler)
    server.serve_forever()

client.py

import time
import socket


def follow(thefile):
    thefile.seek(0, 2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line


def connect_socket():
    HOST, PORT = "localhost", 9999
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    sock.connect((HOST, PORT))
    return sock

if __name__ == '__main__':

    logfile = open("my_log.log")
    loglines = follow(logfile)
    for line in loglines:
        sock = connect_socket()
        # send data
        sock.sendall(bytes(line))

the problem is every time I need to call the connect_socket() method to send a new line.

I'm quite new to this topic so somebody please let me know is there any workaround for this to work in a such a way that once the connection is established between client and server I need to send data continuously to the server without making a new connection again and again.

If I'm connecting only one time and using the same socket object to send data it was throwing

socket.error: [Errno 32] Broken pipe

Some StackOverflow links which I have followed are given below,

1, 2, 3

One thing I found is

Broken Pipe occurs when one end of the connection tries sending data while the other end has already closed the connection.

How can I keep the connection open on both ends? For this use case should I go for an asynchronous method and if so which framework will be the best match tornado or twisted?

TCP socket server not threaded socket server. Most errors is time.sleep() is very bad idea. Server need response for ACK(Don't need any delay) , your code is out of ethernet protocol. You gained 0.5 ms packet succes time if create a true skeleton. Socket isn't unidirectional system, you can't clear hardwre buffer on every packet. My opinion is use WSGI, your project not complex. Check this — dsgdfg

u354356007 u354356007 · Accepted Answer · 2016-10-19T08:28:08

After a line is transmitted, you close the connection on the client, but don't close it on the server.

From the docs:

RequestHandler.finish()

Called after the handle() method to perform any clean-up actions required. The default implementation does nothing.

So you should implement finish as well and close the socket (self.request) there.

Another option is not to close the connection on the client:

sock = connect_socket()
for line in loglines:
    # send data
    sock.sendall(bytes(line))
sock.sendall(b'bye')  # EOF
sock.close()

However in this case you should modify the server code and make sure it can serve multiple clients and it understands when to close the socket. With all these difficulties it is nevertheless the preferable way of doing things. Ideally, a client has a single TCP connection per session for they're costly to establish.

tcp python socket hold connection forever

1 Answers