A server implementing Berkeley's sockets goes through a couple of phases when accepting connections. A lot of the plumbing is generally handled by the framework or the operating system, I'll try pointing them out. I'll try explaining this below with some pseudo-code.
1: Binding to the listening port
The server first asks the kernel to bind to a specific port to start listening on:
void* socket = bind(20);
2: Accepting a connection
This is probably the point that causes some misconceptions. The server gets a connection through the bound socket, but instead of using the listening port (20) to handle the communication with the new client it requests a new (random) port from the kernel to be used for a new socket connection. This is typically handled by the operating system.
void* clientSocket;
// Block until a client connects. When it does,
// use 'clientSocket' (a new socket) to handle the new client.
socket->accept(clientSocket);
// We'll use 'clientSocket' to communicate with the client.
clientSocket.send(someBuffer, ...);
// 'socket' is free again to accept more connections,
// so we can do it again:
void* clientSocket2;
socket->accept(clientSocket2);
// Of course, this is typically done in a loop that processes new connections all the time.
As a summary, what's happening is that the listener socket (20) is used only for accepting new connections. After a client establishes connection, a new socket is created to handle that specific connection.
You can test this by examining the socket connection you get as a client after establishing connection. You'll see that the remote port is not 20 anymore (it will be a random port chosen by the remote server).
All of this is shared by tcp, ftp and any protocol using the sockets protocol under its hood.