Weili Xu

Seeking research opportunities in MLsys
Check out my CV here

I am a sophomore in Computer Engineering, currently pursuing a dual degree from University of Illinois Urbana-Champaign and Zhejiang University.

I am fortunate to collaborate with Wenhao Chai and Enxin Song, working on Efficient Long Video Understanding. We built AuroraLong, a hybrid MLLM that handles hour-long videos on a single consumer GPU while surpassing previous SOTA on multiple video understanding benchmarks such as MLVU, MovieChat-1k and VDC. Our paper is under review for CVPR 2025.

I’m interested in various aspects of machine learning and computer systems:

Hardware-aware efficient algorithms
Exploiting sparsity for training and inference acceleration
Applications of multi-modal (video, audio, text, etc.) long-context modeling

news

Jan 21, 2025	Started to work as Teaching Assistant for ECE 220 Computer Systems & Programming, with Prof. Bruce Yu and Prof. Ujjal Bhowmik
Sep 06, 2024	Started to work as Teaching Assistant for ECE 120 Introduction to Computing, with Prof. Lin Qiu, Prof. Zuozhu Liu, and Prof. Ujjal Bhowmik
May 15, 2024	Joined CVNext Lab, advised by Prof. Gaoang Wang

selected publications

preprint
Bringing RNNs Back to Efficient Open-Ended Video Understanding

Weili Xu, Enxin Song, Wenhao Chai, and 3 more authors

In submission to CVPR 2025 , Nov 2024

Abs Bib PDF Supp

The challenge of long video understanding lies in its high computational complexity and prohibitive memory cost, since the memory and computation required by transformer-based LLMs scale quadratically with input sequence length. We proposed AuroraLong to address this challenge by replacing the LLM component in MLLMs with RWKV, an RNN-like language model that handles input sequence of arbitrary length with constant-size hidden states. To further increase throughput and efficiency, as well as to reduce the gap between RWKV’s 4k context length and the long video token sequence length, we combine visual token merge with linear RNN models by reordering the visual tokens by their sizes in ascending order. AuroraLong shows superior performance on various video benchmarks, for example, obtaining an average accuracy of 87.0 on scene transition in MVBench, beating GPT-4V (83.5) and Gemini Pro (75.4), highliting the possibilities that efficient linear RNNs can democratize long video understanding. To our best knowledge, we are the first to use a non-transformer LLM backbone for video understanding.
@article{xu2024auroralong, title = {Bringing RNNs Back to Efficient Open-Ended Video Understanding}, author = {Xu, Weili and Song, Enxin and Chai, Wenhao and Wen, Xuexiang and Ye, Tian and Wang, Gaoang}, publisher = {preprint}, month = nov, year = {2024}, }