23 March 2023 — NUS Computing Assistant Professor Yang You and Ph.D. students, along with collaborators from Bytedance, has won the Distinguished Paper award for their paper on click-through rate (CTR) model training.
This selection came from the 37th Association for the Advancement of Artificial Intelligence Conference (AAAI-23), which was held between 7-14 February 2023 in Washington, DC, USA. It aims to promote research in artificial intelligence (AI) and foster scientific exchange between researchers, scientists, engineers, and students across AI and its related disciplines.
Dr Yang at the AAAI-23 Conference in Washington, DC, USA
The group’s paper, titled “CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU,” explores the use of large-batch training to accelerate the training process of CTR prediction systems.
CTR prediction estimates the probability of an item on a website being clicked. As mind-blogging amounts of data are generated every day, there is a grave need to ensure the CTR model is kept updated with deep-learning training. The team found that while training with a large batch size significantly improves Graphics Processing Unit (GPU) utilisation and shortens the training time, it also causes the prediction system to lose accuracy.
To tackle the problem, the team proposed a “better hyper-parameter scaling strategy and a column-wise adaptive clipping method (CowClip) to stabilise the training process.”
“As a result, we scaled up the batch size on the Criteo dataset with the DeepFM model to 128K and completed training time in 10 minutes on one V100 GPU,” explained Zangwei on how CowClip can remarkedly reduce training time for CTR model training.
Besides being a highly valuable topic in the industry, he also believes that the team’s theoretical and experimental analyses and proposal, helped their paper stand out.
“Personally, I am surprised and happy. This award has motivated me to continue my research,” expressed Zangwei, the first author of the paper.