AI-based End-to-End and AI-Enhanced Video Coding

Moving Picture, Audio, and Data Coding by Artificial Intelligence is the Standards Developing Organisation whose mission is Data Coding by Artificial Intelligence. Founded in September 2020, it has developed and published 12 technical documents on audio enhancement, human-machine conversation, execution of AI applications, company performance prediction, neural network watermarking, and metaverse model. The talk on MPAI will briefly describe the mission, the organisation, and the scope of activities.

Video internet traffic has grown rapidly, leading to increased strain on infrastructure, slower speeds, and higher costs. Video compression can alleviate this problem; deep learning-based coding tools have shown great potential to reduce bitrate and improve quality. MPAI is working to standardize these tools. In particular, MPAI-EVC is working to improve an existing video codec by 25% in terms of encoding gain by exploiting deep learning.

Recently, neural network for video coding has been an active research topic, receiving attention from both academic society and industry entity. The end-to-end video coding (EEV) of MPAI exploits AI-based data coding technologies in an end-to-end fashion, namely fully neural network based video codec. MPAI-EEV is at the level of Functional Requirements. The essential use cases and related reference model are under construction. This talk elaborates recent advances in EEV and reports the model design, which outperforms the latest H.266/VVC standard in terms of MS-SSIM.