I know that there are clustering algorithms for points obviously, but I have a different scenario. I have many rays, all whose start points are on a sphere in 3D, and whose direction vectors point inwards into the sphere. Some of the rays are pointing towards a point A, others are pointing towards a point B, etc, with some noise(ie the rays don't perfectly intersect each other). Is there a clustering algorithm that will allow me to cluster the rays based on which point they are pointing towards? I don't know the locations of points A, B, etc, only the start points and direction vectors of the rays.
For example, in this picture is a sample setup, but in 2D, and I don't know which rays are red or blue at the start. How would I cluster the rays into red and blue? Or, how would I find the locations of the points they're pointing towards?
One solution I thought of was taking pairs of rays, finding the closest point between those two rays(in 2D this is the point of intersection if you extend the rays), and doing this for every pair of rays(so I'd get n(n-1)/2 points, where n is the number of rays). Then, I could use the regular clustering algorithm on those points. But, that's not seeming to work - I'm getting just one big lump of points at the origin, I don't know why that's happening.