How to work with TCP Sockets in Python (with Select Example)

To create a socket, there is a function called socket. It accepts family, type, and proto arguments (see documentation for details). To create a TCP-socket, you should use socket.AF_INET or socket.AF_INET6 for family and socket.SOCK_STREAM for type.
Here’s a Python socket example:

It returns a socket object which has the following main methods:

• bind()
• listen()
• accept()
• connect()
• send()
• recv()

bind(), listen() and accept() are specific for server sockets. connect() is specific for client sockets. send() and recv() are common for both types. Here is an example of Echo server from documentation:

Here we create a server socket, bind it to a localhost and 50000 port, and start listening for incoming connections. To accept an incoming connection we call accept() method which will block until a new client connects. When this happens, it creates a new socket and returns it together with the client's address. Then, in an infinite cycle, it reads data from the socket in batches of 1024 bytes using method recv() until it returns an empty string. After that, it sends all incoming data back using a convenient method sendall() which inside repeatedly calls send(). And after that it simply closes the client's connection. This example can serve only one incoming connection because it does not call accept() in a cycle.

A client-side code looks simplier:

Here instead of bind() and listen() it calls only connect() and immediately sends data to the server. Then it receives 1024 bytes back, closes the socket, and prints the received data.

All socket methods are blocking. For example, when it reads from a socket or writes to it the program can't do anything else. One possible solution is to delegate working with clients to separate threads. However, creating threads and switching contexts between them is not really a cheap operation. To address this problem, there is a so-called asynchronous way of working with sockets. The main idea is to delegate maintaining the socket's state to an operating system and letting it notify the program when there is something to read from the socket or when it is ready for writing.

There are a bunch of interfaces for different operating systems:

• poll, epoll (linux)
• kqueue, kevent (BSD)
• select (crossplatform)

They are all about the same so let’s create a server using Python select. Here’s a Python select example:

As you can see, there is much more code than in the blocking Echo server. That is primarily because we have to maintain a set of queues for different lists of sockets, i.e. writing, reading, and a separate list for erroneous sockets.

Creating server socket looks the same except for one line: server.setblocking(0). This is done to make the socket nonblocking. This server is more advanced since it can serve more than one client. The main point is in selecting sockets:

Here we call select.select to ask the OS to check given sockets whether they are ready to write, read, or if there is some exception respectively. That is why it passes three lists of sockets to specify which socket is expected to be writable, readable, and which should be checked for errors. This call will block the program (unless a timeout argument is passed) until some of the passed sockets are ready. In this moment, the call will return three lists with sockets for specified operations.

Then it sequentially iterates over those lists and, if there are sockets in them, it performs corresponding operations. When there is the server socket in inputs, it means that a new client has arrived. Therefore, it calls accept(), adds a returned socket to inputs and adds a Queue for incoming messages which will be sent back. If there is another socket in inputs, then some messages have arrived and ready to be read so it reads them and places them into the corresponding queue.

For writable sockets, it gets pending messages (if any) and writes them to the socket. If there is any error in the socket, it removes the socket from the lists.

This is how sockets work at a lower level. However, in most cases, there is no need to implement the logic at such a low level. It is recommended to use some higher level abstractions such as Twisted, Tornado, or ZeroMQ, depending on the situation.