Scikit-Learn DBSCAN creating a cluster with two highly disconnected islands

0
votes

In this case: a picture is worth a thousand words ..

Hello Purple Cluster!

enter image description here

How did this come about? First - to describe the data and the settings for DBSCAN:

  • 100 data points
  • Using a custom distance metric to generate a precomputed distance matrix
  • The custom distance metric is simple: Only consider the r0 (x-distance)

Everything besides that rogue pair of purple points looks precisely as desired. How did that purple cluster jump clear over the Yellow Wall and claim that rogue pair of points on the top left?

Update It has been verified that there are exactly three clusters. I.e. this is not a bug in choosing three colors to denote four clusters. The verification was directly from the dbscan predicted outputs (not some hypothesis of mine):

Here that is:

NumClusters is 3 counts are (array([-1,  0,  1]), array([ 8, 67, 25]))

Another update To clarify: the 2 purple points are being added to the far right cluster (also in purple). They are not a fourth cluster. So the question is - why are those points being added to the furthest away cluster instead of the nearby green and yellow ones?

1
pythonscikit-learndbscan
>>> "it has been verified that there are exactly three clusters". Different algos may have different opinions on what a cluster is. - Sergey Bushmanov
@SergeyBushmanov The verification was via the output generated by DBSCAN : i'm not guessing. Will add that to the question to be clear. - WestCoastProjects
This only means that DBSCAN thinks those two points are different enough to form a separate cluster. - Sergey Bushmanov
@SergeyBushmanov What are you talking about? Those 2 points were added to that cluster to the far right. How does that happen when there are 2 other clusters right next door? You seem to misunderstand that the two points are in some fourth cluster. - WestCoastProjects
It is not me talking about, It's DBSCAN. - Sergey Bushmanov

1 Answers

0
votes

This is interesting. I added the cluster number and we see that for the purple it is -1.

enter image description here

That means NO CLUSTER . So that's how the pair in the upper left can "share" the same "cluster" with the ones on the far right: they do not actually share the cluster but just the identifier that sckit-learn uses for NO cluster.