Publications
(* indicates equal contribution)
2023
X2-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Yan Zeng, Xinsong Zhang, Hang Li, Jiawei Wang, Jipeng Zhang, Wangchunshu Zhou
in Proc. of T-PAMIEvaluate Large Language Models on Controlled Generation Tasks
Jiao Sun*, Yufei Tian*, Wangchunshu Zhou*, Nan Xu*, Qian Hu, John Frederick Wieting, Xuezhe Ma
in Proc. of EMNLP 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan
in Proc. of EMNLP 2023Doolittle: Benchmarks and Corpora for Academic Writing Formalization
Shizhe Diao, Yongyu Lei, Liangming Pan, Tianqing Fang, Wangchunshu Zhou, Sedrick Scott Keh, Min-Yen Kan, Tong Zhang
in Proc. of EMNLP 2023Let’s Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
Ruida Wang, Wangchunshu Zhou, Mrinmaya Sachan
in Proc. of EMNLP 2023 (Findings)To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis
Fuzhao Xue, Yao Fu, Wangchunshu Zhou, Zangwei Zheng, Yang You
in Proc. of NeurIPS 2023Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
in Proc. of ACL 2023 (Findings)Commonsense Knowledge Transfer for Pre-trained Language Models
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
in Proc. of ACL 2023 (Findings)Learning to Predict Persona Information for Dialogue Personalization without Explicit Persona Description
Wangchunshu Zhou, Qifei Li, Chenle Li
in Proc. of ACL 2023 (Findings)EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Tiannan Wang*, Wangchunshu Zhou*, Yan Zeng, Xinsong Zhang
in Proc. of ACL 2023 (Findings)Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Yan Zeng*, Wangchunshu Zhou*, Ao Luo*, Ziming Cheng, Xinsong Zhang
in Proc. of ACL 2023Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, Mrinmaya Sachan
in Proc. of ICML 2023Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Ying Jiao, Kumar Shridhar, Peng Cui, Wangchunshu Zhou and Mrinmaya Sachan
in Proc. of AIED 2023Predicting Reference-Based MT Metrics Without the Reference
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
in Proc. of EACL 2023Write and Paint: Generative Vision-Language Models are Unified Modal Learners
Shizhe Diao, Wangchunshu Zhou, Xinsong Zhang, Jiawei Wang
in Proc. of ICLR 2023
2022
Efficiently Tuned Parameters are Task Embeddings
Wangchunshu Zhou*, Canwen Xu*, Julian McAuley
in Proc. of EMNLP 2022VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
Wangchunshu Zhou*, Yan Zeng*, Shizhe Diao*, Xinsong Zhang*
in Proc. of ICML 2022BERT Learns to Teach: Knowledge Distillation with Meta Learning
Wangchunshu Zhou*, Canwen Xu*, Julian McAuley
in Proc. of ACL 2022Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu*, Wangchunshu Zhou*, Jingjing Xu, Hao Zhou, Lei Li
*in Proc. of ACL 2022
2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Ke Xu, Julian McAuley, Furu Wei
in Proc. of EMNLP 2021 (Oral)Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei
in Proc. of EMNLP 2021Reducing Generic Response and Better Dialogue History Modeling with Inverse Adversarial Training
Wangchunshu Zhou*, Qifei Li*, Chenle Li
in Proc. of ACL 2021 (Oral)Blow the Dog Whistle: A Chinese Dataset for Cant Understandingwith Common Sense and World Knowledge
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Ke Xu, Julian McAuley, Furu Wei
in Proc. of NAACL 2021Pre-training Text-to-Text Transformers for Concept-centric Common Sense
Wangchunshu Zhou*, Dong-Ho Lee*, Ravi Selvam, Seyeon Lee, Bill Yuchen Lin, Xiang Ren
in Proc. of ICLR 2021
2020
Connecting the Dots Between Fact Verification and Fake News Detection
Qifei Li*, Wangchunshu Zhou*
in Proc. of COLING 2020 (Oral)BERT Loses Patience: Fast and Robust Inference with Early Exit
Wangchunshu Zhou*, Canwen Xu*, Tao Ge, Julian McAuley, Ke Xu, Furu Wei
in Proc. of NeurIPS 2020Towards Interpretable Natural LanguageUnderstanding with Explanations as Latent Variables
Wangchunshu Zhou*, Jinyi Hu*, Hanlin Zhang*, Xiaodan Liang, Maosong Sun, Chenyan Xiong, Jian Tang
in Proc. of NeurIPS 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Canwen Xu*, Wangchunshu Zhou*, Tao Ge, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020Scheduled DropHead: A Regularization Method for Transformer Model
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020 (Findings)Improving Grammatical Error Correction with Machine Translation Pairs
Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou
in Proc. of EMNLP 2020 (Findings)Pseudo Bidirectional Decoding for Local Sequence Transduction
Wangchunshu Zhou, Tao Ge, Ke Xu
in Proc. of EMNLP 2020 (Findings)CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi and Xiang Ren
in Proc. of EMNLP 2020 (Findings)Self-Adversarial Learning with Comparative Discrimination for Text Generation
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou
in Proc. of ICLR 2020Learning to Compare for Better Training and Evaluation of Open Domain Text Generation Models
Wangchunshu Zhou, Ke Xu
in Proc. of AAAI 2020 (Oral)