Course Paper L.-K.Hua Seminar July12,2023 2.1.4 YUV Matching However,using the transform and inverse transform formulas directly does not work well,and gets result like figure 8. Figure 8:Matching by YUV if min(min(R))<0 R =R -min(min(R)) end if max(max(R))>256 RR max(max(R))+256; end if min(min(G))<0 G=G-min(min(G)) end if max(max(G))>256 G=G/max(max(G))*256; end if min(min(B))<0 B=B-min(min(B)) end if max(max(G))>256 B =B max(max(B))+256; end What is more if the original chrominances are an the same (for a grayscale image,they are alwavs 128).this matching method can't work on U or V.So,a oftingte pt ofYUV mteing l the s One thought is to use 7
Course Paper L.-K. Hua Seminar July 12, 2023 2.1.4 YUV Matching However, using the transform and inverse transform formulas directly does not work well, and gets result like figure 8. Figure 8: Matching by YUV The problem is, by transforming formula, if R, G and B are all in [0, 255], then Y, U and V are all in [0, 255], but controlling the range of YUV can’t control the range of RGB into uint8. So we need an additional normalization: if min(min(R)) < 0 R = R - min(min(R)); end if max(max(R)) > 256 R = R / max(max(R)) * 256; end if min(min(G)) < 0 G = G - min(min(G)); end if max(max(G)) > 256 G = G / max(max(G)) * 256; end if min(min(B)) < 0 B = B - min(min(B)); end if max(max(G)) > 256 B = B / max(max(B)) * 256; end What is more, if the original chrominances are all the same (for a grayscale image, they are always 128), this matching method can’t work on U or V. So, a right method on YUV must have the ability of distinguishing different colors from the same grayscale. One thought is to use the result of RGB matching as the input of YUV matching like this: function J = match_YUV(T, S) T = match_RGB(T, S); T = to_YUV(T); 7
Course Paper L.-K.Hua Seminar July12.2023 S=to_YUV(S); J=to_RGB(match_RGB(T,S)); end The matching result is like figure 9 Figure 9:Matching by YUV with RGB first While the co does get brighter,the be region caused by the sky in sore image shows again the importance of taking place into account. Figure 10:Grayscale,color style,matching by RGB,and matching by YUV tha by YUV.In all.althor gh YUV mat brigh table is probably not a good choice 8
Course Paper L.-K. Hua Seminar July 12, 2023 S = to_YUV(S); J = to_RGB(match_RGB(T, S)); end The matching result is like figure 9. Figure 9: Matching by YUV with RGB first While the color does get brighter, the blue region caused by the sky in source image shows again the importance of taking place into account. Figure 10: Grayscale, color style, matching by RGB, and matching by YUV In another example shown by figure 10, because the chromatic imbalance, matching by RGB has a better quality than modified by YUV. In all, although YUV matching may let the color brighter, because the serial is not as clear as RGB intensity, use YUV to build a look-up table is probably not a good choice. 8
Course Paper L.-K.Hua Seminar Jly12,2023 2.2 Wavelet Methods 2.2.1 Frequency matural uumber the wte thstanc fuction frequ de(X,T)=>ckllf(x)-f(T)2,d.(X.S)=>skllf(x)-f(S) By experiences,c should be higher in smaller k,while sshould be higher in larger k meaning that low-frequency terms are more likely to become the distance of color styles Assuming fa can variate independently,best f(X)in every level k can be written as a linear combination of f(T)and f(S). So,under this kind of assumption,the result can be written as F-1(a1fi(S)+bifi(T).....axfx(S)+bxfN(T)) Here F-means some kind of Fourier inverse transform. In spite of modifying the parameters,the key question is,how to construct different levels of "Fourier coefficients"that describe the message of an image,and also,how to add information about position to this model. 2.2.2 Wavelet Transform 2-dimensional discrete wavelet transform can get the low-frequency part,vertically high- frequency part,horizontally high-frequency part and diagonally high-frequency part of an image as figure 11 (the image of high frequency is not clear because to scale it in [0,255]is unreasonable),called wavelet coefficients. Figure 11:The low frequency and high frequency part 9
Course Paper L.-K. Hua Seminar July 12, 2023 2.2 Wavelet Methods 2.2.1 Frequency Fourier transform is a well-known method to extract frequency message from a timing signal. If the frequency message of an image can be written as fk(X) (fk(X) is a vector for each natural number k), then we can write the distance function as: dc(X, T) = X k ckkfk(X) − fk(T)k 2 , ds(X, S) = X k skkfk(X) − fk(S)k 2 By experiences, ck should be higher in smaller k, while sk should be higher in larger k, meaning that low-frequency terms are more likely to become the distance of color styles. Assuming fk can variate independently, best fk(X) in every level k can be written as a linear combination of fk(T) and fk(S). So, under this kind of assumption, the result can be written as F −1 (a1f1(S) + b1f1(T), . . . , aN fN (S) + bN fN (T)) Here F −1 means some kind of Fourier inverse transform. In spite of modifying the parameters, the key question is, how to construct different levels of ”Fourier coefficients” that describe the message of an image, and also, how to add information about position to this model. 2.2.2 Wavelet Transform Instead of using simple Fourier transform that loses all place information, wavelet transform is usually a better way to do image-processing [3] . 2-dimensional discrete wavelet transform can get the low-frequency part, vertically highfrequency part, horizontally high-frequency part and diagonally high-frequency part of an image as figure 11 (the image of high frequency is not clear because to scale it in [0, 255] is unreasonable), called wavelet coefficients. Figure 11: The low frequency and high frequency part 9
Course Paper L.-K.Hua Seminar July12.2023 Continuing with the wavelet transformation on the low-frequency term,we can get different layers of wavelet coefficients.As has been required,different layers of coefficients can change independently,and the image can be reconstructed by a sequence of coefficients,so merging different layers of coefficients becomes a feasible option. Denote the wavelet coefficients as h(X),...,(X),(X).Here (X)means the low- frequency term in the last transformation,while h(X)meaning the three high-frequency terms in the first transformation,h2(X)meaning the three high-frequency terms in the second transformation,and so on.Given ak+1 dimensional vector A with each component between 0 and 1,the result can be written as: I=W-l(h1(T)+(1-)h1(S),,Ah(T)+(1-x1)h(S),olk(T)+(1-Ao)l4(S) The following figure 12 shows result when Ao=1 and ;=1 for i>0. Figure 12:The influence of for 6,7,8.9 dimensions From these hard coeficients ()we can see the nature of merging wavelet coefficients.that is.to plit the imag o int and change the color of eac are to the uares are 2.2.3 Soften the Result The result above has clear edges where the square changes.To soften these pseudo-edges, need to be smoother,which means its difference need to be lower. The cases in figure 13 has A of: [0,0.1,0.3,0.5,0.7,0.9,1,1,1,1,1 10
Course Paper L.-K. Hua Seminar July 12, 2023 Continuing with the wavelet transformation on the low-frequency term, we can get different layers of wavelet coefficients. As has been required, different layers of coefficients can change independently, and the image can be reconstructed by a sequence of coefficients, so merging different layers of coefficients becomes a feasible option. Denote the wavelet coefficients as h1(X), . . . , hk(X), lk(X). Here lk(X) means the lowfrequency term in the last transformation, while h1(X) meaning the three high-frequency terms in the first transformation, h2(X) meaning the three high-frequency terms in the second transformation, and so on. Given a k + 1 dimensional vector λ with each component between 0 and 1, the result can be written as: I = W−1 (λkh1(T) + (1 − λk)h1(S), . . . , λ1hk(T) + (1 − λ1)hk(S), λ0lk(T) + (1 − λ0)lk(S)) The following figure 12 shows result when λ0 = 1 and λi = 1 for i > 0. Figure 12: The influence of λ for 6, 7, 8, 9 dimensions From these hard coefficients (λi ∈ {0, 1}), we can see the nature of merging wavelet coefficients, that is, to split the image into squares, and change the color of each square to the style of source image. The higher layers of coefficients are merged, the larger the squares are. 2.2.3 Soften the Result The result above has clear edges where the square changes. To soften these pseudo-edges, λ need to be smoother, which means its difference need to be lower. The cases in figure 13 has λ of: [0, 0.1, 0.3, 0.5, 0.7, 0.9, 1, 1, 1, 1, 1] 10
Course Paper L.-K.Hua Seminar Jmly12,2023 [0,0.2,0.4,0.6,0.8,1,1,1,1,1,1] [0,0.3,0.6,0.9,1,1,1,1,1,1,1] [0,0.4,0.8,1,1,1,1,1,1,1,1] Figure 13:The influence of smoother A Clearly,when becomes smoother,the edges are weakened,but more regional information of source image is displayed in the result,going against the requirement of "color style" Another attempt is to combine merging with matching methods,using merging as a fine tuning after matching. For instance,the result of matching RGB and then let A=(1,0.95,0.8)is figure 14. Figure 14:Combine with matching 11
Course Paper L.-K. Hua Seminar July 12, 2023 [0, 0.2, 0.4, 0.6, 0.8, 1, 1, 1, 1, 1, 1] [0, 0.3, 0.6, 0.9, 1, 1, 1, 1, 1, 1, 1] [0, 0.4, 0.8, 1, 1, 1, 1, 1, 1, 1, 1] Figure 13: The influence of smoother λ Clearly, when λ becomes smoother, the edges are weakened, but more regional information of source image is displayed in the result, going against the requirement of ”color style”. Another attempt is to combine merging with matching methods, using merging as a finetuning after matching. For instance, the result of matching RGB and then let λ = (1, 0.95, 0.8) is figure 14. Figure 14: Combine with matching Another way to smooth the merging result is localized contrast enhancement method, as figure 15. 11