Abstract: Compared with traditional videos, infrared videos in airport surveillance scenario with a large depth of filed consist of smooth background and foreground areas with less textures, thus processing stronger temporal and spatial correlations. Although commonly used video coding tools can be utilized to compress infrared videos, they only utilize local information to reduce the redundancy, resulting in high computational complexity. Based on a global perspective of the data, this paper considers the whole infrared surveillance video as a third-order tensor, then adopts tensor decomposition for approximate representation to reduce the data volume, and finally quantizes the resulting factor matrices to further improve compression efficiency. Meanwhile, CP decomposition and Tucker decomposition are adopted to analyze and compare the compression performance of these two tensor decomposition methods. Experimental results show that, compared with HEVC, Tucker decomposition can achieve about 77.7% BD-rate savings for videos containing lots of smooth areas. For videos with complex content and significant motion, however, both CP decomposition and Tucker decomposition suffer from BD-PSNR lose of over 10 dB. Nevertheless, the encoding time of CP decomposition and Tucker decomposition is just about 0.57% and 2.25% of that of HEVC, respectively.