Publications
(* indicates equal contribution)
2024
Enhancing Data Quality via Training Dynamics from Private Domains for Collaborative Fine-Tuning of Large Language Models
Wanru Zhao, Hongxiang Fan, Shell Xu Hu, Wangchunshu Zhou, Nicholas Donald Lane
in Proc. of NeurIPS 2024PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
Noah Wang, Feiyu Duan, Yibo Zhang, Wangchunshu Zhou, Ke Xu, Wenhao Huang, Jie Fu
in Proc. of EMNLP 2024 (Findings)MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications
Xiangru Tang, Chunyuan Deng, Hanmin Wang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein
in Proc. of EMNLP 2024 DemoHow Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs
Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie
in Proc. of ECCV 2024AutoAct: Automatic Agent Learning from Scratch via Self-Planning
Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, chengfei lv, Huajun Chen
in Proc. of ACL 2024RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
Zekun Wang, Z.Y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Wenhu Chen, Jie Fu, Junran Peng
in Proc. of ACL 2024 (Findings)CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models
Yizhi LI, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Wenhao Huang, Chenghua Lin, Wenhu Chen, Jie Fu
in Proc. of ACL 2024 (Findings)LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild
Ziyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou, Hongxia Yang, Kun Kuang, Fei Wu
in Proc. of ACL 2024 (Findings)OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Fuzhao Xue, Zian Zheng, Yao Fu, Jinjie Ni, Zangwei Zheng, Wangchunshu Zhou, Yang You
in Proc. of ICML 2024Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data
Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein
in Proc. of NAACL 2024 (Oral)SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
Zekun Wang, Jingchang Chen, Wangchunshu Zhou, Haichao Zhu, Jiafeng Liang, Liping Shan, Ming Liu, Dongliang Xu, Qing Yang, Bing Qin
in Proc. of COLING 2024
2023
X2-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Yan Zeng, Xinsong Zhang, Hang Li, Jiawei Wang, Jipeng Zhang, Wangchunshu Zhou
in Proc. of T-PAMIEvaluate Large Language Models on Controlled Generation Tasks
Jiao Sun*, Yufei Tian*, Wangchunshu Zhou*, Nan Xu*, Qian Hu, John Frederick Wieting, Xuezhe Ma
in Proc. of EMNLP 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan
in Proc. of EMNLP 2023Doolittle: Benchmarks and Corpora for Academic Writing Formalization
Shizhe Diao, Yongyu Lei, Liangming Pan, Tianqing Fang, Wangchunshu Zhou, Sedrick Scott Keh, Min-Yen Kan, Tong Zhang
in Proc. of EMNLP 2023Let’s Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
Ruida Wang, Wangchunshu Zhou, Mrinmaya Sachan
in Proc. of EMNLP 2023 (Findings)To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis
Fuzhao Xue, Yao Fu, Wangchunshu Zhou, Zangwei Zheng, Yang You
in Proc. of NeurIPS 2023Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
in Proc. of ACL 2023 (Findings)Commonsense Knowledge Transfer for Pre-trained Language Models
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
in Proc. of ACL 2023 (Findings)Learning to Predict Persona Information for Dialogue Personalization without Explicit Persona Description
Wangchunshu Zhou, Qifei Li, Chenle Li
in Proc. of ACL 2023 (Findings)EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Tiannan Wang*, Wangchunshu Zhou*, Yan Zeng, Xinsong Zhang
in Proc. of ACL 2023 (Findings)Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Yan Zeng*, Wangchunshu Zhou*, Ao Luo*, Ziming Cheng, Xinsong Zhang
in Proc. of ACL 2023Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, Mrinmaya Sachan
in Proc. of ICML 2023Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Ying Jiao, Kumar Shridhar, Peng Cui, Wangchunshu Zhou and Mrinmaya Sachan
in Proc. of AIED 2023Predicting Reference-Based MT Metrics Without the Reference
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
in Proc. of EACL 2023Write and Paint: Generative Vision-Language Models are Unified Modal Learners
Shizhe Diao, Wangchunshu Zhou, Xinsong Zhang, Jiawei Wang
in Proc. of ICLR 2023
2022
Efficiently Tuned Parameters are Task Embeddings
Wangchunshu Zhou*, Canwen Xu*, Julian McAuley
in Proc. of EMNLP 2022VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
Wangchunshu Zhou*, Yan Zeng*, Shizhe Diao*, Xinsong Zhang*
in Proc. of ICML 2022BERT Learns to Teach: Knowledge Distillation with Meta Learning
Wangchunshu Zhou*, Canwen Xu*, Julian McAuley
in Proc. of ACL 2022Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu*, Wangchunshu Zhou*, Jingjing Xu, Hao Zhou, Lei Li
*in Proc. of ACL 2022
2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Ke Xu, Julian McAuley, Furu Wei
in Proc. of EMNLP 2021 (Oral)Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei
in Proc. of EMNLP 2021Reducing Generic Response and Better Dialogue History Modeling with Inverse Adversarial Training
Wangchunshu Zhou*, Qifei Li*, Chenle Li
in Proc. of ACL 2021 (Oral)Blow the Dog Whistle: A Chinese Dataset for Cant Understandingwith Common Sense and World Knowledge
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Ke Xu, Julian McAuley, Furu Wei
in Proc. of NAACL 2021Pre-training Text-to-Text Transformers for Concept-centric Common Sense
Wangchunshu Zhou*, Dong-Ho Lee*, Ravi Selvam, Seyeon Lee, Bill Yuchen Lin, Xiang Ren
in Proc. of ICLR 2021
2020
Connecting the Dots Between Fact Verification and Fake News Detection
Qifei Li*, Wangchunshu Zhou*
in Proc. of COLING 2020 (Oral)BERT Loses Patience: Fast and Robust Inference with Early Exit
Wangchunshu Zhou*, Canwen Xu*, Tao Ge, Julian McAuley, Ke Xu, Furu Wei
in Proc. of NeurIPS 2020Towards Interpretable Natural LanguageUnderstanding with Explanations as Latent Variables
Wangchunshu Zhou*, Jinyi Hu*, Hanlin Zhang*, Xiaodan Liang, Maosong Sun, Chenyan Xiong, Jian Tang
in Proc. of NeurIPS 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020Scheduled DropHead: A Regularization Method for Transformer Model
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020 (Findings)Improving Grammatical Error Correction with Machine Translation Pairs
Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020 (Findings)Pseudo Bidirectional Decoding for Local Sequence Transduction
Wangchunshu Zhou, Tao Ge, Ke Xu
in Proc. of EMNLP 2020 (Findings)CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi and Xiang Ren
in Proc. of EMNLP 2020 (Findings)Self-Adversarial Learning with Comparative Discrimination for Text Generation
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou
in Proc. of ICLR 2020Learning to Compare for Better Training and Evaluation of Open Domain Text Generation Models
Wangchunshu Zhou, Ke Xu
in Proc. of AAAI 2020 (Oral)