Skip to content
Home » Color Palette Extraction and Colored Context

Color Palette Extraction and Colored Context

Who, at first glance at a painting, has never said: “Oh, what beautiful colors!”

How to extract these colors? Do they determine the final impression of the viewer?

Creating different atmospheres from the same colors, and thus arousing different emotions, seems plausible. In the challenge #17, we have already seen the same gray appear dark here, light there, then downright yellowish or bluish in another context. This is not yet an emotion, but it is already a deception: we think we see a color where it is not!

But that with the same pixels – that is to say, exactly the same constituents of the image – we can perceive very different colors, that is the order of magic!

And yet, I invite you to watch this little video (6 minutes) to convince you.

To find the images of the video, go to the section “Rearranging pixels”. You can download them and check that the histograms are indeed identical.

This article is dedicated to Alain Huet, who unfortunately passed away in January 2022. Remember, he is the author of the article: “The colors of master paintings…and yours“. It is therefore a pretext to talk again about the extraction of the color palette of a digital image.

Finding the palette of a painting

Before speculating on the impressions that the colors in a picture might produce, we need to identify them.

In our untimely discussions with Alain, we not only debated the best way to extract these colors but also how to present them. Possibly by limiting ourselves to the main colors.

Let’s take the key image in the video above: the reproduction of the painting “The Wheat Field”, by Vincent Van Gogh.

Reproduction of the painting "The Wheat Field" of V. van Gogh.
Vincent van Gogh, The Wheat Field, 1889 (source: wiki commons)

Utilities at our disposal

Several utilities offer us the possibility to extract the palette of an image. Often this is limited to five colors.

Adobe Color

The tool offered by Adobe is quite satisfactory; compare the palette it extracts from the picture with that of another program, such as

A comparison between two color schemes extracted from the painting "The Wheat Field", by two different tools.
Palette of the painting “The Wheat Field”, extracted by Adobe (left) to compare with a palette extracted by another program (right)

However, we would like to have more colors (a dark green for example) and have an idea of their proportion.

The “dark” option provides another scheme of five colors, including a dark green we were just looking for! In fact, the tool offers five options, each generating a scheme made of five colors. In the case of “The Wheat Field”, there are thirteen colors in total. The table below shows the result of Adobe’s algorithm for each option.

Table showing all colors extracted  by Adobe color tool from the painting "The Wheat Field, according to the option chosen.
Overview of the color scheme extracted from the “Wheat Field”, depending on the option chosen in the Adobe tool. On the first line, all the extracted colors, independently of the option

Of course, you can also choose the colors on the painting itself. So what is the point then? First of all, it allows you to isolate a color from its context, and then, via another tab, to place it on the color wheel – that’s the subject of the “Organizing extracted colors” section.

Really a powerful and efficient tool. In addition itl also allows you to extract gradients. In short, all that’s missing is the proportions…and maybe one or another color. Limiting the choice to five colors is a bit of a stretch! One detail however: the “dark” option aside, the adequacy between the qualifiers (i.e. colorful, bright, muted, deep) and the character of the color scheme remains for me enigmatic. Finally, a button that gives access to all the colors, all options included, would be welcome.

Color Analyzer

The Color Analyzer application fulfills our wish by showing the different percentages of each color, starting with the most important ones. Moreover, the algorithm extracts three main colors: a “base” color, an “accent” color, and an “assort” color.

Color palette extracted from two different reproductions of the same painting, "The Wheat Field" of V. van Gogh
The palette extracted by Color Analyzer on two versions of “The Wheat Field” (original file on left and right)

Here we better understand the adjectives associated with the extracted colors: the “base color” indicates that it is the most important on the image. The term “accent” is well seen to qualify the first color offering a significant contrast with the base color. As for the term “assort” it denotes a color “close” to the base color, at least based on one of its properties of hue, saturation or clarity.

Good reproduction

Whatever the tool, choose a good reproduction; indeed, look how the left and right color schemes above differ! And yet, originally two reproductions of the same painting.

However, despite the differences between the two reproductions, three of the five colors extracted by Adobe (option “colorful”) remain surprisingly close. As for the set of colors extracted via the five options, it confirms the disappearance of a gray-blue that we had on the other reproduction, an important color since it is characteristic of the landscape in the far distance. By the way, doesn’t this gray appear much bluer on the painting?

Comparing Adobe color scheme extraction on two different reproductions of "The Wheat Field"
Comparison of the palettes of the two reproductions of the “Wheat Field” by the Adobe tool. In the upper part, in the form of rectangles, the five colors of the “colorful” option; on top, colors extracted from the greenish reproduction (source); below, that of the other reproduction (source). In the lower part, in the form of dots, all the colors of all options. The upper line shows the “greenish” palette, the lower one , the palette of the other reproduction

It is a pity that Alain Huet, on his chromapicture site where he does chromatic analysis of paintings, did not take the time to choose the digital reproductions. He wrote to me in 2021:

“You are right. In the beginning, I worked only with the mediocre screen of my laptop and I was overly confident in the automatic equalization filter of photopaint (which compensated the grayness of my screen). And I was more busy tweaking my program and website than tweaking the images.”

Do we need a preprocessing?

How to know if the greenish clouds in the second reproduction reflect an artistic choice or result from bad shooting conditions? Should these errors be corrected?

Alain and I had opposite views on this question. According to Alain, degradation, shadows, yellowing, lighting, justify a pre-processing. He writes on his chromapicture site:

“[…] many paintings appear yellowed due to the ageing of the varnish or poor shooting conditions. It is therefore useful to use photo editing software to refresh the colors. There is obviously no guarantee that the result will be perfectly in line with the original work, but the improvement is comparable to painting restorations by specialized workshops.”

Personally, I am opposed to any preprocessing. Indeed, it can radically change the atmosphere of a painting, and a fortiori its palette. Moreover, we don’t have the skills of painting restorers, who are more knowledgeable about the type of degradation suffered. In my opinion, the idea is not to extract the “original” colors but those of a digital image. And if there is preprocessing, the user must be able to compare the transformed image with the original. This is the case on Alain’s site: a click on the image gives access to the unprocessed version.

Organizing extracted colors

How to present the colors once extracted from the image?

The choice of Color Analyzer is to present first the three main colors next to the image, then all the colors according to their importance, indicating their percentage. This choice avoids classifying the colors on the basis of their properties, but makes any comparison difficult.

Adobe Color

By clicking on the tab “Chromatic Wheel”, we visualize, on a chromatic disc, the five extracted colors. One can thus compare their hue and their saturation. Indeed, the closer the colors are to the center, the more they will resemble a gray. The chromatic disc is however always the same, so that dark nuances may be missed because of a too big contrast.

Adobe color tool presentation of the extracted color scheme using the color wheel tab.
Presentation of the “colorful” and “dark” color scheme extracted from the “Wheat Field” (from:wiki commons) via the Color Wheel tab.

However, the information about the proportions of the colors is still missing.


On his site chromapicture, Alain Huet proposes an original organization.

Each color is represented by a colored disk, more or less large depending on the proportion of color in the image. The user thus perceives its relative importance at first glance. The disks are distributed according to their hue on a white disk representing a blank color wheel. The agglomeration of the dots is enough to guess the main colors present.

However, unlike most graphic tools, the dots are off-center according to their degree of lightness and not according to their color intensity (i.e. saturation). As a result, the colored grays are found at the periphery of the disk; the white balance even becomes visible: see the accumulation of gray-greens for the “Wheat Field” palette extraction – Alain having chosen this reproduction. The disadvantage, however, is the placement of perfectly neutral tones: having no defined hue, they will be arbitrarily placed on the disk. Most graphic applications avoid this pitfall since the neutrals are all placed in the center.

In short, the organization of colors is intuitive (see the central disk below) and efficient.

Screenshot of Chromapicture site showing a preprocessed  reproduction of "The Wheat Field", and the extracted palette.
Screenshot of the analysis of the painting “The Wheat Field” by chromapicture. On the left, the preprocessed image; in the middle, the extracted palette distributed on a chromatic disc; on the right, all the extracted colors organized by “families”.

Since the most abundant colors are placed first, the superposition in principle lets guess the underlying colors. Obviously, if two colors present in the same proportions differ only by their saturation, only one will be visible. So Alain imagined presenting each color in another format: a series of colored rectangles organized by “family”; their surface reflects their proportion in the painting.

This representation of colored disks of varying size stacked on an empty chromatic wheel is really ingenious.


In this article, I have deliberately ignored tools that present colors in a three-dimensional space. These, like Colororate (see challenge #4) for example, require rotating the axes to see the distribution of colors.

Although limited to a flat representation, the three tools Adobe Color, Color Analyzer, and chromapicture offer three very different views, each with their advantages and disadvantages.

By taking colors out of their context, we necessarily put them in another: whether on a white disk (chromapicture), a color wheel (Adobe Color), or next to each other (Color Analyzer, Adobe Color and chromapicture).

These visualizations do not use the same conventions: on the color wheel, sometimes yellow is placed at noon (Adobe Color), sometimes at two o’clock (chromapicture). As many proposals as there are color wheels! With Alain, we also discussed the choice of a color space. Alain had tested two of them on a set of paintings (see the central window, below).

Also, in an earlier version of the site, Alain had opted for a presentation in the form of colored bands rather than rectangles stacked like towers. I have a preference for this first layout, do you?

Alain Huet's experiences using two different color spaces to display the color palette of the painting "Millstones, middle of the day"
Excerpt from an exchange with Alain Huet about the choice of the color space (central window). In the right part, the colors are arranged in bands rather than in towers. This arrangement was used in the older versions of the Chromapicture site. On the left, a reproduction of “Millstones, middle of the day” by Claude Monet.

Let’s dream

The ideal tool should allow us to switch from one presentation to another via a tab.

Sometimes, we are surprised to discover such or such color in the extracted palette. Then we wonder where the algorithm found it. A tab that would show all the pixels identified as being of this or that color would be welcome.

In fact, the palette extraction process hides a step of reducing the set of colors. This step is called “color quantization”. To what extent does this step preserve our perception of the image? How does look the painting once each pixel has been assigned a color of the palette? We should have access to this result. Color quantization is usually done after averaging colors over a set of pixels. Therefore, very colorful small elements may easily disappear or be attenuated by this process. The orange of the poppies, however important in the perception of the painting, will probably have disappeared in this stage of color quantization.

Finally, I would like to have a tool that separates neutral and near-neutral tones. Image processing software often distinguishes between “dark”, “medium” and “light” tones; I would like to see a distinction between “neutral”, “near-neutral” and the others. This specific visualization would allow one to grasp at first glance the white balance and appreciate the more subtle nuances of these versatile colors.

Rearranging pixels

For Alain, the arrangement of the colors extracted from the images was of little importance. All were equal. This position has never ceased to challenge me. The perception of a color, even presented in a palette, is influenced by the colors that surround it.

To convince you of this, I made the video in the introduction. I reorganized the pixels of the reproduction of the “Wheat Field” according to different rules. The conditions of the experiment are different, however; here I did not apply any quantization nor averaging: I kept the image size and pixel exact color values.

You can download these images to compare their respective histograms: they are identical!

The images of the video

The pixels of the "Wheat field" have been randomly rearranged on each column to produce this "Beach" image
“Beach”, based on “The Wheat Field” by V. van Gogh (V. Lacroix CC-BY-SA).
The pixels in each column of the reproduction have been redistributed randomly, then rotated.
The pixels of the "Wheat field" have been randomly rearranged to produce this "Wet sand" image
Wet Sand,” based on “The Wheat Field” by V. Van Gogh (V. Lacroix CC-BY-SA).
The pixels of the reproduction have been redistributed randomly.

The pixels of the "Wheat field" have been rearranged to generate this "Weaving" image
“Weaving”, V. Lacroix, after “Field of Wheat” by V. van Gogh (V. Lacroix CC-BY-SA)
A reorganization of the pixels that separates the different shades of the painting.
The pixels of the "Wheat field" have been rearranged to generate this "Crop" image
“Crops”, V. Lacroix, after “Field of wheat” by V. van Gogh (V.Lacroix CC-BY-SA)
Another reorganization of the pixels that separates the different shades of the painting.
The pixels of the "Wheat field" have been rearranged to generate this "Drips" image
“Drips”, V. Lacroix, after “The Wheat Field” by V. van Gogh (V.Lacroix CC-BY-SA)
The darkest pixels are at the top of the image and placed according to their hue.
The pixels of the "Wheat field" have been rearranged to generate this "Watercolor composition" image
“Watercolor composition”, V. Lacroix, after “The Wheat Field” by V. van Gogh (V. Lacroix CC-BY-SA)
The pixels are organized according to their hue, lightness and saturation.
The pixels of the "Wheat field" have been rearranged to generate this "Composition in gray and yellow" image
“Composition in gray and yellow”, V. Lacroix, after “The Wheat Field” by V. van Gogh (V. Lacroix CC-BY-SA)
The pixels are organized so that their color is drowned in gray or yellow.
The pixels of the "Wheat field" have been rearranged to generate this "Sad Field" image
“Sad Field”, V. Lacroix, after “The Wheat Field” by V. van Gogh (V. Lacroix CC-BY-SA)
A reorganization of the pixels that roughly keeps their clarity (value).

To conclude

Knowing the colors constituting an image does not allow to anticipate the colored perception which will result from it. This one depends in particular on the size of the elements and the neighboring colors.

However, the applications that allow to extract the colors present in a picture, an image, are useful to study the color content, to generate inspiring palettes and analyze our perception.

Your favorite color scheme

What is your favorite color scheme? Which color tool do you use to extract the palette of an image? Share them in the comments. And to visualize the whole colors? What do you think of the representations of the different tools? What would be the best way to present the colors of a digital image?

Have you ever been surprised to learn that an image had a color that you had completely missed? Or to realize that an area of the image took on a completely different hue when isolated from its context?

Observe how the context, the distance, the adaptation to a given light influences your perception of colors, let yourself be surprised.

Leave a Reply

Your email address will not be published. Required fields are marked *