Let me first clarify the problem statement. Check out this tweet:
https://twitter.com/jungledragon/status/926894337761345538
Next, click the image itself within the tweet. In the light box that appears, the menu bar below it takes on a meaningful color that is based on the actual pixels in the image itself. Even in this stress test, this is a difficult image given all the light pixels, does it do a fine job in picking an overall color that 1) represents the content of the image 2) is dark/contrasty enough to place white text on it:
I was simultaneously implementing a similar system before I even knew Twitter had this. Check out a preview below:
The examples in the screenshot are optimistic, as there are plenty of situations where the background is too light. Even in seemingly positive examples as seen in my screenshot, most of the time it does not pass the AA or AAA contrast check.
My current approach:
- One-time per image, a JS runs that calculates the average color of all pixels in the image. Note that the average color is not necessarily a meaningful color, such as in the edge case of the spider where the average would be close to white.
- I store the RGB value in the database
- Upon rendering the page (server-side) I dynamically set the background color of the image's caption using a formula
My formula is to convert the RGB to HSL, and then to manipulate in particular the S and L values. Given them a notch, using min/max values to set a treshold. I've tried countless combinations.
Yet it seems like a never-ending struggle because color darkness and contrast are subject to human perception.
Hence my curiosity on how Twitter seems to have nailed this, in particular two aspects:
- Finding a meaningful subject color (not the same as average or dominant color)
- Toning that meaningful color in a way that it remains recognizable (hue) yet is contrasty enough to place light text on it, whilst passing at least the AA contrast check.
I've searched around, but cannot find any information on their implementation. Anybody aware of they do it? Or other proven methods to solve this puzzle end-to-end?