Pan Hui

University of Helsinki - Finland

Keynote presentation on Friday, Oct.11

Practical Edge-assisted Mobile Computing: The Cases of Augmented Reality and Virtual Reality


Smartphone is now a necessity of our daily lives. However, the resource and communication limitations of a single mobile device make it insufficient for satisfying the real-time and interactive constraints of some computation intensive applications, such as mobile Augmented Reality (MAR) and mobile Virtual Reality (MVR). To bridge the gap, we utilize the processing power of edge servers via task offloading and build practical mobile systems which significantly outperforms state-of-the-art mobile systems in terms of scalability, battery life, latency, quality of experience (QoE). For MAR, large-scale object recognition is an essential but time-consuming task. To offload the object recognition task and enhance the system performance, we explore how the GPU and the multi-core architecture on the edge servers would accelerate the large-scale object recognition process. With the carefully designed offloading pipeline and edge acceleration, we are able to finish the whole AR pipeline within one frame interval (33 ms for a 30 fps camera system) while maintaining high recognition accuracy with large-scale datasets. For MVR, existing 360 video streaming systems are suffering from insufficient pixel density, as the video resolution falling within the user’s field of view (FoV) is relatively low. We utilize the edge server for ultra-high-resolution video transcoding and implement a system which streams tile-based viewport adaptive 360 videos onto the mobile clients. With this edge proxy, we successfully achieve 16k 360 video streaming onto off-the-shelf smartphones, achieving high quality frames and fluent 30 fps playback without overwhelming the processing capacity of the smartphones.


Pan Hui received his PhD from the Computer Laboratory at University of Cambridge, and both his Bachelor and MPhil degrees from the University of Hong Kong. He is the Nokia Chair Professor in Data Science and Professor of Computer Science at the University of Helsinki, and the director of the HKUST-DT Systems and Media Lab at the Hong Kong University of Science and Technology. He was an adjunct professor of social computing and networking at Aalto University from 2012 to 2016. He was a senior research scientist and then a Distinguished Scientist for Telekom Innovation Laboratories (T-labs) Germany from 2008 to 2015. His industrial profile also includes his research at Intel Research Cambridge and Thomson Research Paris from 2004 to 2006. He has published more than 250 research papers and with over 15,500 citations. He has 30 granted and filed European and US patents in the areas of augmented reality and mobile computing. He has founded and chaired several IEEE/ACM conferences/workshops, and has been serving on the organising and technical program committee of numerous top international conferences including ACM SIGCOMM, Multimedia, MobiSys, WWW, IEEE Infocom, ICNP, ICDCS, IJCAI, AAAI, and ICWSM. He has been an associate editor for the leading journals IEEE Transactions on Mobile Computing fsince 2014. He was also an associate editor for the IEEE Transactions on Cloud Computing, and a guest editor for IEEE Journal on Selected Areas in Communications, IEEE Communication Magazine, and ACM Transactions on Multimedia Computing, Communications, and Applications. He is a Fellow of The Institute of Electrical and Electronics Engineers (IEEE), and an ACM Distinguished Scientist recognized by the Association for Computing Machinery, the world’s largest scientific and educational computing society.