Masking the JPEG Fourier coefficients


We consider the following simple geometric image of original size 280x280 that we show in double sized to appreciate the details later. The thin line is 1 pixel wide in the original.

line_hv

In the code below, the matrix M is a mask for the Fourier coefficients (actually DCT coefficients) of each 8x8 block, the basic unit in JPEG format. If an entry is 1 the corresponding Fourier coefficient is preserved and if it is 0 it is suppressed. The marked part is modified accordingly in each example with a new definition of the matrix. The original choice M = ones(8); means to leave the original image

jpeg3_1
The a_00 Fourier coefficient gives the average of the function. If we only select it we see the blocks as big pixels.
 %%%%%%%%%%%%%%%%%%%%%%%%
M = zeros(8);
M(1,1) = 1;
%%%%%%%%%%%%%%%%%%%%%%%%

jpeg3_2
Now we select only the first column. It means that we can analyze perfectly the parts not depending on the first coordinate. Then horizontal lines in each block are preserved and vertical lines are averaged as before.
 %%%%%%%%%%%%%%%%%%%%%%%%
M = zeros(8);
M(:,1) = 1;
%%%%%%%%%%%%%%%%%%%%%%%%

jpeg3_3
Taking out  the coefficient a_00 and leaving the rest is like subtracting the average. We only see the boundary blocks.
 %%%%%%%%%%%%%%%%%%%%%%%%
M = ones(8);
M(1,1) = 0;
%%%%%%%%%%%%%%%%%%%%%%%%

jpeg3_4
We can exaggerate this effect suppressing more small frequency harmonics.
%%%%%%%%%%%%%%%%%%%%%%%%
M = ones(8);
M(1:5,1:5) = 0;
%%%%%%%%%%%%%%%%%%%%%%%%

jpeg3_5
This is selecting the northwest triangle of size 8 in the matrix of Fourier coefficients, this is almost one half of the coefficients, exactly 36/64=9/16. Some artifacts appear near boundaries but the quality is not bad taking into account that we are simply deleting coefficients everywhere. In JPEG, near the boundary the high frequencies would not be completely lost.
 %%%%%%%%%%%%%%%%%%%%%%%%
n = 8;
M = ones(8);
M = tril(M,n-8);
M = rot90(M,-1);
%%%%%%%%%%%%%%%%%%%%%%%%

jpeg3_6jpeg3_6
This is an exaggeration of the  previous situation. With the same code a triangle with only 1+2+3=6 coordinates is chosen. The artifacts are now evident but we the proportion of the coefficients is 6/64, meaning that less than the 10% of the information is kept. Imagine the result if you delete 90% of the pixels instead of acting on the Fourier coefficients.
The image to the right is at original size to appreciate the actual quality.
 %%%%%%%%%%%%%%%%%%%%%%%%
n = 3;
M = ones(8);
M = tril(M,n-8);
M = rot90(M,-1);
%%%%%%%%%%%%%%%%%%%%%%%%

See this for another example.



The code

The plots were done with the following Matlab code. It does not work in my version of Octave.


imag = imread('line_hv.png');
imag = rgb2gray(imag);
imag = im2double(imag);

figure(1)
imshow( imag )

%%%%%%%%%%%%%%%%%%%%%%%%
% MODIFY THIS PART
%
M = ones(8);
%%%%%%%%%%%%%%%%%%%%%%%%

figure(2)
M
img = mask_image(imag, M);
imshow( img )
function [img] = mask_image(imag, M)
T = dctmtx(8);
% Apply DCT
dct_fun = @(block_struct) T * block_struct.data * T';
img = blockproc(imag, [8 8], dct_fun, 'PadPartialBlocks',true);

% Apply mask on Fourier coefficients
mask_fun = @(block_struct) M .* block_struct.data;
img = blockproc(img, [8 8], mask_fun, 'PadPartialBlocks',true);

% Apply inverse DCT
idct_fun = @(block_struct) T' * block_struct.data * T;
img = blockproc(img, [8 8], idct_fun, 'PadPartialBlocks',true);
end
Main program
 Function mask_image