Lu Wei

Biography

I am interested in designing mathematical models and machine learning algorithms for solving fundamental data processing problems, with a specific focus on natural language processing (NLP). I am particularly interested in fundamental topics around modern language models. Prior to joining SUTD, I worked as a postdoctoral research associate at the University of Illinois at Urbana-Champaign and was a PhD student at the Singapore-MIT Alliance of the National University of Singapore. Even earlier, I received my undergraduate degree (1st class honors) in Computer Science from the School of Computing, National University of Singapore.

News

• Sep 2024: Thanks to HTX for awarding a new industry collaboration project to our StatNLP Group (S$1.5M). We look forward to working together!
• Jul 2024: Congratulations to Zhijiang Guo for winning SUTD Outstanding Thesis Award!
• Jun 2024: Congratulations to Zhijiang Guo for securing an Assistant Professor position at HKUST (GZ)!
• May 2024: Congratulations to Guangtao Zeng for securing an internship offer from MIT (with Joshua Tenenbaum)!
• Apr 2024: Honored to be appointed an ST Engineering Distinguished Professor. Thank you, ST Engineering!
• Feb 2024: Excited to assume the role of Editor-in-Chief for Computational Linguistics! Thank you, ACL Executive Committee, for giving me this opportunity to serve our community!
• Jan 2024: Congratulations to Peiyuan Zhang, Guangtao Zeng, and Tianduo Wang for completing the pre-training of our TinyLlama! It’s now trending #1 on Hugging Face (as of 3 Jan 2024).
• Dec 2023: Congratulations to Xiaobing Sun and Jiaxi Li for receiving an Outstanding Paper Award at EMNLP!
• Jul 2023: Congratulations to Sicong Leng for winning the Area Chair Award at ACL, with multiple nominations (top 1.6% papers received nominations)!
• Apr 2023: Honored to be recognized as a Notable AC for ICLR 2023 (1 of 26, out of 410 Area Chairs)! Look forward to continuing to contribute!
• Feb 2023: Congratulations to Jiale Han for joining TU Darmstadt as a Postdoc (with Iryna Gurevych)!
• Dec 2022: Congratulations to Allan Jie for joining UNC Chapel Hill as a Postdoc (with Mohit Bansal)!
• Aug 2022: Thanks to MOE for the AcRF Tier 2 Award to our StatNLP Group (S$930K)!
• Jul 2022: Congratulations to Xinyu Wang for winning the Best Paper Award (top 0.45%=1/221) at SemEval 2022!
• Jun 2022: Our StatNLP Group’s 10-year NLP research is ranked 12th (adj #: 17.8) globally based on CSRankings.
• May 2022: Congratulations to Shengqiang Zhang for joining LMU Munich as a PhD student (with Hinrich Schütze)!
• Feb 2022: Congratulations to Xinyu Wang for winning SemEval 2022 Task 11 in 10 out of 13 tracks!
• Jan 2022: Congratulations to Yuxiang Zhou for joining Warwick University as a Postdoc (with Yulan He)!
• Jul 2021: Congratulations to Jierui Li for joining the University of Texas at Austin as a PhD student (with Raymond Mooney)!

Recent Services

• Editor-in-Chief: Computational Linguistics (2024-)
• PC Co-Chairs: NLPCC 2022, AACL-IJCNLP 2023
• Senior Area Co-Chairs: EMNLP 2021, ACL 2022, NAACL 2022, ACL 2024, EMNLP 2024, ICLR 2025, NAACL 2025
• Area Co-Chairs: ACL 2016, EMNLP 2019, ACL 2020, IJCAI 2021, NAACL 2021, ACL 2021, AAAI 2022, EMNLP 2022, AAAI 2023, ICLR 2023, ACL 2023, NeurIPS 2023, EMNLP 2023, AAAI 2024, ICLR 2024
• General Chair: The 2018 Singapore Symposium on NLP (SSNLP)
• Executive Committee Member: Association for Computational Linguistics (ACL) (2024-2029)
• Executive Committee Member: Asian Federation of Natural Language Processing (AFNLP) (2022-2024)

Recent & Selected Publications

• Unraveling Feature Extraction Mechanisms in Neural Networks
  Xiaobing Sun, Jiaxi Li and Wei Lu
  EMNLP 2023
  Outstanding Paper Award (“Interpretability, Interactivity, and Analysis of Models for NLP” Track)
• Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parsers
  Jiaxi Li and Wei Lu
  ACL 2023
• Learning Multi-Step Reasoning by Solving Arithmetic Tasks
  Tianduo Wang and Wei Lu
  ACL 2023
• One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning
  Guangtao Zeng, Peiyuan Zhang and Wei Lu
  ACL 2023
• Tell2Design: A Dataset for Language-Guided Floor Plan Generation
  Sicong Leng, Yang Zhou, Mohammed Haroon Dupty, Wee Sun Lee, Sam C. Joyce and Wei Lu
  ACL 2023
  Area Chair Award, with multiple award nominations: top 1.6% submissions.
• Tab-CoT: Zero-shot Tabular Chain of Thought
  Ziqi Jin and Wei Lu
  ACL 2023 (Findings)
• Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning
  Zhanming Jie and Wei Lu
  ACL 2023 (Findings)
• Sampling Better Negatives for Distantly Supervised Named Entity Recognition
  Lu Xu, Lidong Bing and Wei Lu
  ACL 2023 (Findings)
• Differentiable Data Augmentation for Contrastive Sentence Representation Learning
  Tianduo Wang and Wei Lu
  EMNLP 2022
• Better Few-Shot Relation Extraction with Label Prompt Dropout
  Peiyuan Zhang and Wei Lu
  EMNLP 2022
• Unsupervised Non-transferable Text Classification
  Guangtao Zeng and Wei Lu
  EMNLP 2022
• Implicit N-grams Induced by Recurrence
  Xiaobing Sun and Wei Lu
  NAACL 2022
  Revealing the intrinsic relationship between (non-Markovian) RNNs and (Markovian) classical n-grams
  Provides some explanations for observations made years ago in the community (e.g., this and this)
• Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction
  Zhanming Jie, Jierui Li and Wei Lu
  ACL 2022
  Towards system 2 learning with explainable deductive reasoning
  Top 5% among all submissions (according to released ARR statistics)
• A Knowledge-based System for Multilingual Named Entity Recognition
  Xinyu Wang, Yongliang Shen, Jiong Cai, Tao Wang, Xiaobin Wang, Pengjun Xie, Fei Huang, Weiming Lu, Yueting Zhuang, Kewei Tu, Wei Lu, Yong Jiang
  SemEval 2022
  Best Paper Award (top 0.45% submissions: 1/221; Best system in 10 out of 13 tracks; totally 49 teams worldwide)
• Mixed Cross Entropy Loss for Neural Machine Translation
  Haoran Li and Wei Lu
  ICML 2021
  Selected as a long oral paper (top 3% submissions; A new way of training “non-standard” structured prediction models)
• Grarep: Learning graph representations with global structural information
  Shaosheng Cao, Wei Lu and Qiongkai Xu
  CIKM 2015
  Most cited CIKM paper within a 5-year window (twice: 2014-2019, 2015-2020)
• A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
  Wei Lu and Hwee Tou Ng
  EMNLP 2011
  Best Paper Award (top 0.16% submissions: 1/628)

Full publication list

Group Members

• Haoran Li (PhD student; from: University of Electronic Science and Technology of China)
• Guangtao Zeng (PhD student; from: Sun Yat-Sen University, China; AI Singapore Scholarship)
• Yao Xiao (PhD student; from: Shanghai Jiao Tong University, China; A*STAR SINGA Scholarship)
• Tianduo Wang (PhD student; from: Singapore University of Technology and Design; 1st Class Honors; Singapore Computer Society Excellence Award)
• Jiaxi Li (PhD student; from: Singapore University of Technology and Design; 1st Class Honors)
• Calvin Tan (PhD student; from: Nanyang Technological University, Singapore)
• Chen Huang (PhD student; from: East China Normal University, China)
• Haong Phi Long (PhD student; from: Hanoi University of Science and Technology, Vietnam)
• Peiwen Zhao (Visiting PhD student; from: Beijing Institute of Technology, China)
• Yutian Lin (Research Assistant; from: Singapore University of Technology and Design)
• Yanbao Wang (Research Assistant; from: Singapore University of Technology and Design)
• John-Henry Lim (UROP; from: Singapore University of Technology and Design)

Group Alumni (PhD Students)

• Zhijiang Guo
  Now: University of Cambridge (UK)
  Internship: University of Edinburgh
• Yanyan Zou
  Now: JD (Beijing)
  Internship: Microsoft Research Asia
• Hao Li
  Now: Tencent (Singapore)
  Internship: Alibaba DAMO
• Allan Jie
  Now: ByteDance (Singapore)
  Internship: Allen Institute for Artificial Intelligence
  ISTD Best PhD Thesis Award
• Lu Xu
  Now: Huawei (Singapore)
  EDB IPP Industry PhD Program
• Liying Cheng
  Now: Alibaba (Singapore)
  EDB IPP Industry PhD Program
• Richard Sun
  Now: A*STAR (Singapore)
  EMNLP 2023 Outstanding Paper Awardee

Group Alumni (Others)

• Aaron Liang (Intern; from: River Valley High School, Singapore)
• Shichen Li (Visiting fellow; from: Soochow University, China)
• Renhang Liu (UROP; from: Singapore University of Technology and Design)
• Ziyu Ge (UROP; from: Singapore University of Technology and Design)
• Jerry Feng (UROP; from: Singapore University of Technology and Design)
• Yijia Chen (UROP; from: Singapore University of Technology and Design)
• Pratyush Saini (Intern; from: Indian Institute of Technology, Delhi, India)
• Ruoxi Ning (Visiting fellow; from: Zhejiang University, China)
• Peiyuan Zhang (Research assistant; to: Nanyang Technological University)
• Sicong Leng (Research assistant; to: Nanyang Technological University)
• Zechen Li (Visiting fellow; from: Northeastern University, USA)
• Shuibai Zhang (Visiting fellow; from: Westlake University, China)
• Shukai Ma (Research assistant; from: University of Melbourne, Australia)
• Qian Yang (Visiting fellow; from: Harbin Institute of Technology (Shenzhen), China)
• Jiale Han (Visiting PhD student; to: Technische Universität Darmstadt, Germany)
• Yilin Liu (Visiting fellow; from: University of Southern California, USA)
• Vanessa Tan Yun Han (Research assistant; from: Singapore University of Technology and Design)
• Zhiyuan Zeng (Visiting fellow; to: University of Zurich, Switzerland)
• Zhehao Zhang (Visiting fellow; from: Shanghai Jiao Tong University (Zhiyuan College), China)
• Xinyu Wang (Visiting PhD student; from: ShanghaiTech University, China)
• Shengqiang Zhang (Research assistant; to: Ludwig Maximilian University of Munich, Germany)
• Zizhe Jiang (Visiting fellow; to: University of Texas at Austin, USA)
• Shuyi Jia (UROP; to: Georgia Institute of Technology, USA)
• Christine Liu (Visiting fellow; from: University College London, UK)
• Qingcheng Zeng (Visiting fellow; to: Northwestern University, USA)
• Yuxiang Zhou (Visiting PhD student; to: University of Warwick, UK)
• Jierui Li (Research assistant; to: University of Texas at Austin, USA)
• Jiaqi Zeng (Visiting fellow; to: Carnegie Mellon University, USA)
• Xiaobao Wu (Visiting fellow; from: Tsinghua University, China)
• Ankit Garg (Visiting fellow; to: University of California San Diego, USA)
• Rui Qiao (Research assistant; to: National University of Singapore)
• Jue Wang (Visiting fellow; from: Zhejiang University, China)
• Xingce Bao (Visiting fellow; from: École Polytechnique Fédérale de Lausanne, Switzerland)
• Ivan Sekulić (Visiting fellow; from: Heidelberg Institute for Theoretical Studies, Germany)
• Jack Wu (Visiting fellow; from: York University, Canada)
• Jieyu Chen (Visiting fellow; to: Cornell University, USA)
• Boyuan Zhuang (Research assistant; from: University of Science and Technology of China)
• Jing Liao (Visiting student; from: Zhejiang University, China)
• Ke Shi (Research assistant; to: A*STAR AI, Singapore)
• Jiaqi Pan (Visiting fellow; to: Carnegie Mellon University, USA)
• Rishabh Bhardwaj (Visiting student; to: National University of Singapore)
• Leiming Du (Research assistant; to: Huawei, China)
• Yu Jian Wu (Visiting student; to: Columbia University, USA)
• Anh Nguyen Tuan (Visiting fellow; from: Vietnam National University)
• Peter Phandi (Research assistant; from: National University of Singapore)
• Utkarsh Sinha (Visiting fellow; to: Columbia University, USA)
• Yiran Wang (Research officer; to: Columbia University, USA)
• Amila Silva (Research assistant; to: Singapore Management University)
• Hans Du (Visiting student; from: Academia Sinica, Taiwan)
• Bill Lin (Visiting student; to: University of Southern California, USA)
• Yinan Shao (Visiting student; from: Harbin Institute of Technology, China)
• Bailin Wang (Visiting student; to: University of Edinburgh, UK)
• Rachael Boyle (Visiting student; from: University of California, Berkeley, USA)
• Yaqi Xie (UROP; to: National University of Singapore)
• Siyuan Wang (UROP; to: Carnegie Mellon University, USA)
• Aldrian Obaja Muis (Research assistant; to: Carnegie Mellon University, USA)
• Raymond Hendy Susanto (Research assistant; from: National University of Singapore)
• Swee Kiat Lim (Reserach officer; from: National University of Singapore)
• Wei Yang (Visiting student; to: University of Waterloo, Canada)
• Hesam Amoualian (Visiting student; from: University of Grenoble Alpes, France)
• Shaosheng Cao (Research assistant; to: AI Department, Ant Financial Services Group/Alipay, China)
• Liling Tan (Research assistant; from: Saarland University, Germany)
• Moein Khajehnejad (Research officer; from: Sharif University of Technology, Iran)
• Jonathan Löfgren (Research officer; from: Uppsala University, Sweden)
• Bojia Qiu (Visiting student; from: Zhejiang University (Chu Kochen Honors College))
• Wenqiang Xu (Research officer; from: University of Science and Technology of China (Special Class for the Gifted Young))
• Tu Thanh Vu (Research officer; to: University of Massachusetts Amherst, USA)
• Si Kai Lee (Research officer; to: Columbia University, USA)
• Sadegh Charmchi (Research officer; to: University of Alberta, Canada)
• Ruidan He (UROP; to: National University of Singapore)
• Cuong Xuan Chu (Research officer; to: Max Planck Institute, Germany)
• Lifei Chen (Research officer; to: École Polytechnique Fédérale de Lausanne, Switzerland)
• Sean Matthew English (Research officer; to: Fraunhofer-Gesellschaft Institute, Germany)

Recent Research Grants

• Principal Investigator, Alibaba Innovative Research Program (2018-2019), US$100K
• Principal Investigator, MOE Tier 2 Grant (2018-2021). S$540K
• Principal Investigator, DSO Collaboration Grant (2017-2019). S$500K
• Principal Investigator, Defence Innovative Research Programme Project (TBA). S$890K
• Principal Investigator, MINDEF Grant – CISDeM T3 (2014-2018). S$610K

Awards

• Area Chair Award, The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), 2023
• Best Paper Award, the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP’11), 2011
• Dean’s Graduate Research Excellence Award, School of Computing, National University of Singapore, 2009-2010
• Research Achievement Award, School of Computing, National University of Singapore, 2008-2009
• President’s Fellowship, National University of Singapore, 2006-2007
• Singapore-MIT Alliance Scholarship, 2005-2006, 2006-2009
• Dean’s List, School of Computing, National University of Singapore, 2001-2005
• Singapore MOE Undergraduate Scholarship, 2000-2005
• 1st Prize (perfect score: 140/140), National Mathematical Olympiad Competition, China, 1996
• 1st Prize, National Mathematical Olympiad Competition, China, 1995

Teaching

• 2020 Summer: 50.040: Natural Language Processing (Undergraduate)
  Students’ Feedback
• 2019 Fall: 01.112: Machine Learning (Undergraduate)
  Students’ Feedback
• 2019 Summer: 50.040: Natural Language Processing (Undergraduate)
  Students’ Feedback
• 2018 Fall: 50.570/51.504: Machine Learning (Graduate)
• 2018 Fall: 01.112: Machine Learning (Undergraduate)
• 2018 Spring: 01.112: Machine Learning (Undergraduate)
  Students’ Feedback
  
• 2017 Fall: 50.570: Machine Learning (Graduate)
  Students’ Feedback
• 2017 Fall: 01.112: Machine Learning (Undergraduate)
  Students’ Feedback
  
• 2016 Fall: 50.007: Machine Learning (Undergraduate)
  Students’ Feedback
• 2016 Spring: 50.570: Machine Learning (Graduate)
  Students’ Feedback
• 2015 Fall: 50.007: Machine Learning (Undergraduate)
  Students’ Feedback
• 2015 Spring: 50.570: Machine Learning (Graduate)
  (no survey)
• 2014 Fall: 50.007: Machine Learning (Undergraduate)
  Students’ Feedback
• 2014 Spring: 50.570: Machine Learning (Graduate)
  Students’ Feedback