Deep Learned Image Compression and Semantic Segmentation Networks

Abstract

Deep learning neural networks have become the mainstream in many computer vision tasks such as image recognition and semantic segmentation. To reduce storage space and transmission bandwidth, the images are often needed to be compressed. However, compressed images might greatly affect the performance of semantic segmentation. To balance both needs, it is necessary to optimize segmentation neural networks for specific compression algorithms with suitable compression rates. Recently, deep learning networks for image compression have been greatly improved and better than traditional coding standards. Therefore, in this paper, we propose a neural network that can adjust the compression ratio, allowing end-to-end training of image compression and semantic segmentation modules. The proposed transformer-based compression network uses the prompt tuning method to adjust the compression ratio. For semantic segmentation tasks, which take compressed image features as the input, the modified Mask2Former architecture can perform the segmentation directly. Experimental results show that the proposed method can reduce the performance loss in semantic segmentation tasks caused by image compression.