The Hough transformation and the probabilistic Hough transformation use the same technique to detect lines in an image. i.e. both calculate r and theta values. The difference between the two is the set of edge points that each algorithm uses. The probabilistic Hough transformation randomly samples from the set of all the edge points and uses them to detect lines, whereas the Hough transformation uses all the edge points in an image.
Now, this is my answer to your question. In the probabilistic Hough transformation, we take any two points (x1,y1) and (x2,y2) from the set of randomly selected edge points and calculate (a,b) using the following equations.
y1 = x1(a) + b
y2 = x2(a) + b
A pair of (a,b) essentially represents the line joining the two points (x1,y1) and (x2,y2). In the code, we maintain a linked list that stores these (a,b) pairs and a count value associated with the pair. We calculate (a,b) values for all possible pairs from the selected edge points. While calculating (a,b), we know corresponding edge points. In order to know which edge points contribute to each line, we can store these edge points. Then, we can compute the endpoints of each line in the image using this information.
Reference: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV1011/macdonald.pdf