publications

2026

1. EMNLP Main

   Rethinking LoRA Memory Through the Lens of KV Cache Compression

   Chunsheng Zuo, Liaoyaqi Wang, William Jurayj, and 2 more authors

   2026
2. COLM

   Process Supervision of Confidence Margin for Calibrated LLM Reasoning

   Liaoyaqi Wang, Chunsheng Zuo, William Jurayj, and 2 more authors

   2026
3. KDD

   ARIA: A Causal-Aware Framework for Rescuing LLM Reasoning in Trustworthy Materials Discovery

   Yi* Cao, Liaoyaqi* Wang, Jieneng Chen, and 3 more authors

   In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, Republic of Korea, 2026

   Abs DOI

   Generative models have revolutionized the process of materials discovery, yet they often fail to satisfy underlying physical causality. Through an analysis of Large Language Models (LLMs) augmented with knowledge graphs derived from current literature, we uncover a phenomenon termed contextual tunneling, where models ”over-anchor” on narrow, retrieved evidence while suppressing global physical reasoning. To address this problem, we introduce ARIA, a causal-aware framework that conditions knowledge use on mechanistic completeness. ARIA routes each query through a three-tier cascade: (i) direct causal reasoning when complete evidence chains of Process-Structure-Property (PSP) are available, (ii) physics-informed analogical transfer for sparse or novel material systems, and (iii) explicit parametric fallback when external evidence is incomplete. As a proof of concept, we construct a Knowledge Graph (KG) containing 2,839 extracted PSP relations from peer-reviewed articles in the materials literature and evaluate ARIA on forward prediction and inverse design tasks for two-dimensional (2D) materials. ARIA mitigates contextual tunneling, improves over unaugmented and naive KG-augmented baselines, and provides further gains when an online literature search is used for evidence enrichment. Crucially, ARIA produces auditable causal traces, enabling physically grounded and trustworthy AI-assisted materials discovery.

2025

1. COLM

   Always Tell Me The Odds: Fine-grained Conditional Probability Estimation

   Liaoyaqi Wang^*, Zhengping Jiang^*, Anqi Liu, and 1 more author

   In Second Conference on Language Modeling, 2025
2. EMNLP Findings

   LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts

   Wenhao* Zheng, Liaoyaqi* Wang, Dongshen Peng, and 5 more authors

   In Findings of the Association for Computational Linguistics: EMNLP 2025, Nov 2025

   Abs DOI

   Clinical trials are pivotal yet costly processes, often spanning multiple years and requiring substantial expenses, motivating predictive models to identify likely-to-fail drugs early and save resources. Recent approaches leverage deep learning to integrate multimodal data for clinical outcome prediction; however, they rely heavily on manually designed modality-specific encoders, limiting their adaptability to new modalities and ability to effectively share information across modalities. To address these challenges, we propose a multimodal mixture-of-experts (LIFTED) framework. Specifically, LIFTED transforms modality-specific data into natural language descriptions, encoded via unified, noise-resilient encoders. A sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities, extracting consistent representations. Finally, another mixture-of-experts module dynamically integrates these modality representations, emphasizing critical information. Experiments show that LIFTED significantly outperforms baseline methods in predicting clinical trial outcomes across all phases, highlighting the effectiveness of our proposed approach.

2024

1. EMNLP Findings

   GuardEmb: Dynamic Watermark for Safeguarding Large Language Model Embedding Service Against Model Stealing Attack

   Liaoyaqi Wang and Minhao Cheng

   In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

   Abs DOI

   Large language model (LLM) companies provide Embedding as a Service (EaaS) to assist the individual in efficiently dealing with downstream tasks such as text classification and recommendation. However, recent works reveal the risk of the model stealing attack, posing a financial threat to EaaS providers. To protect the copyright of EaaS, we propose GuardEmb, a dynamic embedding watermarking method, striking a balance between enhancing watermark detectability and preserving embedding functionality. Our approach involves selecting special tokens and perturbing embeddings containing these tokens to inject watermarks. Simultaneously, we train a verifier to detect these watermarks. In the event of an attacker attempting to replicate our EaaS for profit, their model inherits our watermarks. For watermark verification, we construct verification texts to query the suspicious EaaS, and the verifier identifies our watermarks within the responses, effectively tracing copyright infringement. Extensive experiments across diverse datasets showcase the high detectability of our watermark method, even in out-of-distribution scenarios, without compromising embedding functionality. Our code is publicly available at \urlhttps://github.com/Melodramass/Dynamic-Watermark.