Our visual system attracts many researchers including medical, vision, optical, psychological, and color scientists. For clinical applications^[1,2], medical, vision, and optical scientists investigated image formation by our eyes to correct dysfunctions such as astigmatism, presbyopia, and myopia. On a functional level, vision and color scientists try to understand how we generate color sensations from received images and try to model phenomena such as color difference^[3], chromatic adaptation^[4–6], and color appearance^[7–9]. In this Letter, we model chromatic adaptation in order to predict corresponding colors when the predominant viewing environment is changed.

A chromatic adaptation transform (CAT) is capable of predicting corresponding colors. A pair of corresponding colors consists of a color observed under one illuminant (say, D65) and another color that has the same appearance when observed under a different illuminant (say, A). CATs are part of color appearance models (CAMs)^[7–9], which are important for many industrial applications. These transforms have been extensively studied over several decades ever since Johannes von Kries^[10] in 1902 laid down the foundation for modeling chromatic adaptation. Rather than giving a specific set of equations for the modeling, he instead simply outlined his hypothesis in words and described the potential impact of his ideas. Based on his hypothesis, chromatic adaptation in the visual system is considered the independent change in responsivity of the three types of cone photoreceptors. To present the von Kries hypothesis in terms of a chromatic adaptation model, we need a 3 by 3 matrix M, which transforms the tristimulus values (TSVs) Xβ,Yβ,Zβ under an illuminant called β into the cone-like or sharper sensor spaces (R,G,B or L,M,S spaces). Here, we will use the R,G,B notation. Thus, we have (RβGβBβ)=M(XβYβZβ),(1)where the matrix M can be the well-known HPE matrix^[7], the CAT02 matrix^[8], or the CAT16 matrix^[9]. The entire chromatic adaptation is completed in the R,G,B space. The signals Rβ,Gβ,Bβ are considered to be the initial cone responses. According to the von Kries proportionality law, the von Kries post-adaptation signals Ra,β,Ga,β,Ba,β are given by (Ra,βGa,βBa,β)=(kR,βRβkG,βGβkB,βBβ),(2)where the subscript a signifies adaptation, β represents the illuminant, and R,G,B indicate different channels. The von Kries adaptation factors or coefficients kR,β,kG,β,kB,β are independent of each other and are given by kR,β=1Rw,β,kG,β=1Gw,β,kB,β=1Bw,β,(3)where the subscript w signifies the sensor space signals transformed from the TSV of the illuminant β white point, (Rw,βGw,βBw,β)=M(Xw,βYw,βZw,β),(4)and Xw,β,Yw,β,Zw,β are the TSVs of the illuminant β white point. Thus, if two stimuli sβ and sδ are viewed under illuminants β and δ, respectively, and they are perceived with the same appearance, then we must have (Ra,βGa,βBa,β)=(Ra,δGa,δBa,δ)or(kR,βRβkG,βGβkB,βBβ)=(kR,δRδkG,δGδkB,δBδ).(5)

When Eq. (5) holds, the two stimuli are called corresponding colors.

Note that when we say stimulus sβ in the TSV space, we mean that sβ is a column vector formed by TSVs Xβ,Yβ,Zβ. In this case, sβ can be written as sXYZ,β. When we say stimulus sβ in the cone-like space, we mean that sβ is a column vector formed by cone sponse signals Rβ,Gβ,Bβ obtained using Eq. (1). Similarly, in this case, sβ can be written as sRGB,β. If we let diag (a,b,c) be a 3 by 3 diagonal matrix, the von Kries transform in the cone-like space, denoted by Γδ,β, can be simply defined by Γδ,β=diag(kR,βkR,δkG,βkG,δkB,βkB,δ),(6)and the real von Kries transformation from stimulus sβ to stimulus sδ is a simple matrix and vector multiplication: sRGB,δ=Γδ,βsRGB,βorsXYZ,δ=M−1Γδ,βMsXYZ,β.(7)

Note that the order of the symbols δ,β in the subscript of the von Kries transform Γδ,β is important. Here, δ,β mean that the von Kries transform maps stimulus sβ under illuminant β to stimulus sδ under illuminant δ. Similarly, transform Γβ,δ maps stimulus sδ under illuminant δ to stimulus sβ under illuminant β. Note also that, if two stimuli sβ and sδ are corresponding colors, then sδ and sβ are also corresponding colors, with this property being called symmetry. Thus, we expect the von Kries transform to satisfy this property. In fact, it can be verified that Γδ,βΓβ,δ=I3,(8)where I3 is the 3×3 identity matrix. Equation (8) shows that the von Kries transform has the property of symmetry, as desired. Also, if sβ and sδ are corresponding colors, and sγ and sδ are corresponding colors too, then sγ and sβ must be corresponding colors, and this property is known as transitivity. Similarly, we also expect the von Kries transform to have transitivity. Fortunately, it is indeed the case, since Γγ,δΓδ,β=Γγ,β.(9)

The von Kries transform can be further modified by introducing the modified von Kries adaptation factors: kR,β′=kR,βqR,β,kG,β′=kG,βqG,β,kB,β′=kB,βqB,β.(10)

Based on the above new von Kries adaptation factors, we can have the modified von Kries transform, Γδ,β′, which is defined by Γδ,β′=diag(kR,β′kR,δ′kG,β′kG,δ′kB,β′kB,δ′).(11)

It can be shown that the modified von Kries transform also satisfies the symmetry and transitivity.

Note that if the scaling factors qR,β, qG,β, qB,β in Eq. (10) are all equal to one, the modified von Kries transform becomes the classical von Kries transform, i.e., Γδ,β′=Γδ,β. In fact, by different choices of the scaling factors qR,β, qG,β, qB,β, the modified von Kries adaptation factors become some available adaptation factors in the literatures, such as the Fairchild factors (see page 177 in Ref. [11]) with qR,β=pR,β,(12)pR,β=1+(Lβ)1/3+rE,β1+(Lβ)1/3+1/rE,β,(13)rE,β=3Rw,β/RERw,β/RE+Gw,β/GE+Bw,β/BE;(14)CMCCAT2000^[12], CAT02^[13], and CAT16^[4,9] factors with qR,β=qG,β=qB,β=Yw,β;(15)or Smet et al.^[5,6] factors with qR,β=RE,qG,β=GE,qB,β=BE.(16)Here, RE, GE, BE are obtained using Eq. (1) with the TSVs of the equal energy illuminant white point. Lβ is the luminance of the adapting field and is about 20% of the absolute luminance of the illuminant β. Thus, it can be seen the modified von Kries adaptation factors [Eq. (10)] are more general. We will discuss next how to choose the factors qR,β, qG,β, qB,β. Firstly, they should satisfy qR,βqR,δ=qG,βqG,δ=qB,βqB,δ=c;(17)in this case, it can be shown from Eq. (11) that Γδ,β′=c·diag(Rw,δRw,βGw,δGw,βBw,δBw,β)=cΓδ,β.(18)

Hence, sXYZ,δ=M−1Γδ,β′MsXYZW,β=csXYZW,δ.(19)

Thus, if the CAT maps the TSVs of Xw,β,Yw,β,Zw,β, the output is cXw,δ,cYw,δ,cZw,δ, which is correct as expected. The CAT should make the chromaticity correct as discussed in the Letter^[14] by Hunt et al. However, one may think it is better if the CAT can make the luminance correct as well. It is clear the constant c must be one under this condition, which one may want if the input is Xw,β,Yw,β,Zw,β and then the output is Xw,δ,Yw,δ,Zw,δ. So, the next condition for factors qR,β, qG,β, qB,β is qR,βqR,δ=qG,βqG,δ=qB,βqB,δ=c=1.(20)

The condition of Eq. (20) means factors qR,β, qG,β, qB,β are independent of illuminant. From Eqs. (12)–(14), the Fairchild factors are illuminant dependent, and they may not satisfy conditions of Eqs. (17) and (20). Factors qR,β, qG,β, qB,β defined by Eq. (15) are illuminant dependent, but they satisfy condition Eq. (17). They also satisfy condition Eq. (20) if Yw,β=Yw,δ. Factors qR,β, qG,β, qB,β defined by Eq. (16) use a fixed illuminant; hence, then they will satisfy both conditions of Eqs. (17) and (20).

Up to now, it seems that if the factors qR,β, qG,β, qB,β satisfy the condition of Eq. (20), the modified von Kries transform is in fact the classical von Kries transform. Yes, it is the case. However, one will see the reason for introducing the modified von Kries transform.

However, neither the (classical) von Kries [see Eq. (6)] nor the modified von Kries transform [see Eq. 11)] with factors qR,β, qG,β, qB,β given by any set of Eqs. (12)–(16) shows a tight fit with the experimental visual data sets on corresponding colors (see test results below). To solve this problem, researchers have proposed various linear and nonlinear extensions, as detailed by Fairchild^[11]. The linear extensions related to the International Commission on Illumination (CIE) CAMs^[8,9], such as CAT02^[13] and CAT16^[9], with factors qR,β, qG,β, qB,β defined by Eq. (15), can be expressed as Γδ,β,CATxx=DxxΓδ,β′+(1−Dxx)I3,(21)where xx in the subscript can be 02 for CAT02^[13] and 16 for CAT16^[9]; although, in fact, D02 and D16 are the same. The incomplete adaptation factor Dxx is between 0 and 1. When Dxx is one, Γδ,β,CATxx becomes Γδ,β′ in such a way that CAT02 and CAT16 can be considered as extensions to the modified von Kries transform. However, when Dxx is different from 1 or 0, they no longer satisfy the symmetry and transitivity properties. That is, in general, Γδ,β,CATxx does not satisfy Eqs. (8) and (9). Hence, an inverse CAT is needed for Γδ,β,CATxx, which is simple for linear CATs and is given by (Γδ,β,CATxx)−1 mapping stimulus sδ to stimulus sβ. The CAT Γδ,β,CATxx is normally called a one-step CAT, which directly maps stimulus sβ to stimulus sδ.

Recently, Smet et al.^[5,6] [using factors qR,β, qG,β, qB,β defined by Eq. (16)] and Li et al.^[4] [using factors qR,β, qG,β, qB,β defined by Eq. (15)] proposed two-step CATs via the intermediate CIE illuminant E, which is defined by the equi-energy spectrum (see Fig. 1 in Ref. [5]). Firstly, a one-step CAT such as ΓE,β,CATxx is applied to map stimulus sβ to stimulus sE. In this stage, the adaptation to the illuminant β for our visual system is referred to as illuminant E. Similarly, for the adaptation to the illuminant δ in the second stage, the illuminant E is also used, and a one-step CAT (ΓE,δ,CATxx)−1 is applied to map stimulus sE to stimulus sδ. The end result is the two-step CAT, denoted by Πδ,β,2Step, defined by Πδ,β,2Step=(ΓE,δ,CATxx)−1ΓE,β,CATxx.(22)

Note that the incomplete adaptation factor Dxx in each of the one-step CATs in Eq. (22) may be different. Fortunately, the two-step CAT satisfies the symmetry and transitivity properties^[4]. Furthermore, the two-step CAT performs equally well or better than the one-step CAT in predicting the visual datasets on corresponding colors^[4]. However, the derivation of the two-step CAT is debatable. Why does the adaptation for our visual system always refer to an illuminant (illuminant E) that does not exist in the real world? We should recall that the derivations of the von Kries and modified von Kries transforms do not need an intermediate illuminant.

Can we have a CAT that satisfies symmetry and transitivity without referring to an intermediate illuminant and fits the visual datasets as good as or better than the one-step CAT? The answer is yes. To this end, we have introduced the incomplete adaptation factor D into the modified von Kries adaptation factors rather than into the modified von Kries transform Γδ,β′ [see Eq. (21)]. Thus, the new incomplete adaptation factors under illuminant β are $$\begin{gathered} kR,β″=DβkR,β′+(1−Dβ), \\ kG,β″=DβkG,β′+(1−Dβ), \\ kB,β″=DβkB,β′+(1−Dβ). \end{gathered}$$(23)

The new incomplete adaptation factors under illuminant δ can be similarly defined. As with the derivation of the von Kries or the modified von Kries transform, we have a new CAT, called the generalized von Kries (GvK) transform, which is denoted as Γδ,β″ and uses the new incomplete adaptation factors defined in Eq. (23). Thus, the GvK transform Γδ,β″ is given by Γδ,β″=diag(kR,β″kR,δ″kG,β″kG,δ″kB,β″kB,δ″).(24)

It can be shown that Γδ,β″ satisfies Eqs. (8) and (9). Thus, the GvK transform indeed satisfies the properties of symmetry and transitivity.

Note that the GvK transform has two adaptation factors, Dβ and Dδ. The Dβ factor in CAT02 and CAT16 depends only on the luminance level of illuminant β, and hence, Dβ and Dδ are the same if the luminance levels of the two illuminants are the same. Recently, several papers^[5,6,15] have reported that the D factor affects the performance of CATs and have guessed that the D factor may also depend on correlated color temperature (CCT).

Note also that when we consider the von Kries, modified von Kries, and GvK transforms in TSV space, an associated matrix M mapping the stimulus in TSV to the cone-like space [see Eq. (1)] is necessary. For example, as noted before, the von Kries transform in TSV space is simply given by (M−1Γδ,βM), where the matrix M can be the CAT02, CAT16, or HPE matrices.

Before we test the performance of the GvK transform, we need also to specify the factors qR,β, qG,β, qB,β in Eq. (10). According to the discussion above, it is better that they are independent of illuminant. A simple choice is qR,β=qG,β=qB,β=c2.(25)

Here, c2 is a constant again. According to Eqs. (23) and (24), performance of the GvK transform is also dependent on c2. For the testing below, c2 is set to be 100. The reason for it is explained below.

Performance of the proposed von Kries transform Γδ,β″ with the CAT02, CAT16, and HPE matrices has been tested using the available corresponding color datasets^[16–22], which were used for developing CAT02 and CAT16. The formula employed for the D factor was the one used for CAT02 and CAT16. Comparisons with the von Kries transform Γδ,β, (one-step) CAT02, and CAT16 were also made. The results found are summarized in Table 1, in terms of mean, weighted mean, minimum (Min), and maximum (Max) CIELAB color differences between the predicted and experimental TSVs for each pair of corresponding colors in datasets. There are 21 datasets, and each dataset has a different number of pairs of corresponding colors. The values in the row of Table 1 labeled with Mean are the average of the mean color differences for the different datasets, while the values in the row labeled with Weighted Mean are the weighted mean color difference; the weight for each dataset is the ratio of the number of pairs in this dataset and the number of pairs in all the datasets. The values in the second to last row are the Max of Max color differences for the different datasets. The values in the last row are the Min of Min color differences for the different datasets. The lower the values in Table 1, the better the performance of the corresponding model.

First, Table 1 indicates that when using any of the three matrices, the proposed GvK transform (see results under columns Γδ,β″) is better than the von Kries transform (see results under columns Γδ,β). Note the von Kries and modified von Kries transforms are the same since factors qR,β, qG,β, qB,β satisfy Eq. (25). Second, the proposed GvK transform is equally well as or better than the (one-step) CAT02 with one exception being under Min measure with negligible 0.1 color difference unit (see results under column CAT02) and (one-step) CAT16 (see results under column CAT16). Third, both the von Kries and the proposed GvK transforms perform best using the CAT02 matrix, second best using the CAT16 matrix, and worst using the HPE matrix. However, we should note that the CAT02 matrix has the “yellow–blue” and “purple” problems^[23–25]. The CAT16 matrix^[9,26] was derived for the aim of fitting visual datasets and overcoming the “yellow–blue” and “purple” problems. Therefore, we recommend that the CAT16 matrix should be used for the von Kries, modified von Kries, and proposed GvK transforms.

Note that if the D factors Dβ and Dδ are set to one, any c2 value does not affect the performance of the GvK transform. In fact, in this case, it is simply the von Kries transform, i.e., Γδ,β″=Γδ,β′=Γδ,β. However, when we use the D factor to be the D factor of CAT02 (CAT16), the c2 value indeed affects the performance of the GvK transform. It was found that when c2=100, the GvK transform performs the best; when c2 deviates from 100, the GvK transform performs worse. This may come from three facts. First, all visual datasets tested here, Yw=100. Second, the Yw factor was introduced into the CAT02, CAT16, and CMCCAT2000 for being consistent with the nonlinear CMCCAT97^[27], which was built in CIECAM97s^[7]. As discussed above, the Yw factor is just related to the scaling factors qR,β, qG,β, qB,β [see Eqs. (10), (15)]. Finally, the matrix and D factor of CAT02/CAT16 were derived based on fitting all the visual datasets^[16–22] as best as possible. Hence, it is recommended to use the c2=100 together with the D factor of the CAT02/CAT16 for the GvK transform before a better D factor is developed.

When one of the two illuminants is illuminant E, the GvK transform Γδ,β″ becomes a one-step CAT [see Eq. (21)] with c2=Yw=100. In fact, since the CAT02, CAT16, and HPE matrices are normalized according to the illuminant E, we have Rw,E=Gw,E=Bw,E=Xw,E=Yw,E=Zw,E=100.(26)

Thus, from Eqs. (3), (10), and (14), we have kR,E′=kG,E′=kB,E′=kR,E″=kG,E″=kB,E″=1.(27)

Therefore, from Eq. (15), considering also Eqs. (11), (14), and (17), $$\begin{gathered} ΓE,β″=diag(kR,β″kR,E″kG,β″kG,E″kB,β″kB,E″) \\ =diag(kR,β″kG,β″kB,β″) \\ =Dβdiag(kR,β′kG,β′kB,β′)+(1−Dβ)I3 \\ =DβΓE,β′+(1−Dβ)I3. \end{gathered}$$

Hence, if we let Dβ=Dxx in Eq. (12), we have ΓE,β″=ΓE,β,CATxx.(28)

Equation (28) means that the proposed GvK transform mapping stimulus under illuminant β to stimulus under illuminant E is just the normal one-step CAT from stimulus under illuminant β to stimulus under illuminant E. Remember that the one-step CAT ΓE,β,CATxx is used in the forward mode in CIECAM02/CAM16. Similarly, it can be proved that Γδ,E″=(ΓE,δ,CATxx)−1.(29)

Equation (29) means that the GvK transform mapping stimulus under illuminant E to stimulus under illuminant δ is just the inverse of the normal one-step CAT from the stimulus under illuminant δ to the stimulus under illuminant E. We should bear in mind that the inverse of one-step CAT ΓE,δ,CATxx is used in the inverse mode in CIECAM02/CAM16. Therefore, we conclude that the proposed GvK transform can be used in the current CIECAM02/CAM16.

In conclusion, the von Kries transform was reviewed, and then the modified von Kries transform was derived based on the modified von Kries adaptation coefficients [see Eq. (10)]. The factors qR,β, qG,β, qB,β in Eq. (10) were shown to be better if they satisfy condition Eq. (20), resulting in it being better if the Yw factor in CAT02, CAT16, and CMCCAT2000 is a constant of 100. The Yw factor was introduced into CMCCAT2000 first, later into CAT02 and CAT16 to be consistent with CIECAM97s and CMCCAT97, and further justification was discussed in the Letter given by Hunt et al.^[14]. Since 2000, there was a debate about the Yw factor. There is nothing wrong with a CAT including the Yw factor like CMCCAT2000 and CAT02, since the main purpose of a CAT should make the chromaticity correct. However, when the Yw factor is fixed to a constant like 100, the CAT can make both chromaticity and luminance correct.

It was found that the current linear CAT02 and CAT16 can be considered to be the extension of the modified von Kries transform. However, while the von Kries and modified von Kries transforms satisfy symmetry and transitivity, CAT02 and CAT16 do not satisfy these two properties in general. Finally, a GvK transform has been proposed. The proposed GvK transform, similar to the von Kries and modified von Kries transforms, satisfies the symmetry and transitivity of the properties. Performance evaluation using the available visual datasets^[16–22] showed that the proposed GvK transform performs better than the von Kries and modified von Kries transforms and performs equally well as or better than the (one-step) CAT02 and CAT16. Furthermore, the proposed GvK transform does not need an inverse transform and can be used in CIECAM02/CAM16.

Finally, we note that recently Kerouh et al.^[28] used a CAT to convert the (input) image of a scene captured under one illuminant to the (output) image of the same scene captured under another illuminant. Their results have shown that the CAT affects image content such as edges, texture, and homogeneous area differently. Image-content-based CATs were developed. Comparison results based on multispectral images have shown that the image-content-based CATs perform better than other CATs including the von Kries and Bradford transforms. Our proposed GvK model is evaluated here using the visual corresponding color datasets and may be further evaluated in the future using image data.