Jin Li
Reliable and Efficient Multimodal Intelligence
I study reliable and efficient multimodal systems across language, vision, and speech, with a particular interest in model evaluation, inference-time optimization, and controllable generation. My work spans human-aligned VLM evaluation, lightweight LLM inference control, audio-text alignment, and efficient model deployment.
News
- 2026.09 OpenVTON-Bench was accepted to the NeurIPS 2026 ED Track, focusing on high-resolution controllable virtual try-on evaluation.
- 2026.06 KFC-KWS was accepted as an Oral presentation at Interspeech 2026. This work studies user-defined keyword spotting with CTC-guided keyframe fusion.
- 2026.01 Spectral Logit Sculpting was accepted to IEEE ICASSP 2026. The paper explores inference-time optimization for controlled text generation.
- 2025.08 NC-KWS was accepted to NCMMSC 2025, bringing neural-collapse-inspired ideas to few-shot class-incremental keyword spotting.
- 2025.05 MRIE was accepted to IEEE Access. This work investigates two-stage multimodal learning for rainfall intensity estimation.
Education
Hangzhou Dianzi University
Sep 2022 - Jun 2026
B.Eng. in Electronic Information Engineering · GPA: 89.75 / 100
Key coursework: Machine Vision, Signal Detection and Processing, DSP and Intelligent Systems, Digital Signal Processing, Microelectronics, and EDA Technology.
Sep 2022 - Jun 2026
B.Eng. in Electronic Information Engineering · GPA: 89.75 / 100
Key coursework: Machine Vision, Signal Detection and Processing, DSP and Intelligent Systems, Digital Signal Processing, Microelectronics, and EDA Technology.
Research & Internship Experience
Zhejiang University Binjiang Institute
Research Intern · May 2025 - Aug 2025
Research on LLM reasoning and inference-time optimization.
Research Intern · May 2025 - Aug 2025
Research on LLM reasoning and inference-time optimization.
Intelligent Information Processing Lab, Hangzhou Dianzi University
Undergraduate Researcher · Dec 2023 - Sep 2025
Research on edge intelligent signal processing and multimodal command recognition.
Undergraduate Researcher · Dec 2023 - Sep 2025
Research on edge intelligent signal processing and multimodal command recognition.
Publications
* equal contribution · † corresponding author / project lead
OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation
Jin Li*, Tao Chen*, Kai Wen, Siqi Yin, Shuai Jiang, Weijie Wang, Jingwen Luo, Chenhui Wu†.
A nearly 100K-pair high-resolution virtual try-on benchmark with semantic balancing, dense garment captions, VLM-as-a-Judge evaluation, and representation-based structural metrics aligned with human preference.
NeurIPS 2026 ED Track [arXiv] [code]
Jin Li*, Tao Chen*, Kai Wen, Siqi Yin, Shuai Jiang, Weijie Wang, Jingwen Luo, Chenhui Wu†.
A nearly 100K-pair high-resolution virtual try-on benchmark with semantic balancing, dense garment captions, VLM-as-a-Judge evaluation, and representation-based structural metrics aligned with human preference.
NeurIPS 2026 ED Track [arXiv] [code]
KFC-KWS: Keyframe Fusion with CTC for User-Defined Keyword Spotting
Jin Li, Wenbin Jiang, Ji Hu.
Uses CTC peaky posterior distributions to select phoneme keyframes, aligning audio, phoneme, and text modalities for discriminating highly confusable user-defined keywords.
Interspeech 2026 Oral [arXiv]
Jin Li, Wenbin Jiang, Ji Hu.
Uses CTC peaky posterior distributions to select phoneme keyframes, aligning audio, phoneme, and text modalities for discriminating highly confusable user-defined keywords.
Interspeech 2026 Oral [arXiv]
Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
Jin Li, Zhebo Wang, Tianliang Lu, Mohan Li, Wenpeng Xing, Meng Han.
An inference-time optimization method that uses spectral analysis, entropy statistics, and adaptive low-rank logit transformation to improve controlled text generation without parameter updates.
ICASSP 2026 [arXiv]
Jin Li, Zhebo Wang, Tianliang Lu, Mohan Li, Wenpeng Xing, Meng Han.
An inference-time optimization method that uses spectral analysis, entropy statistics, and adaptive low-rank logit transformation to improve controlled text generation without parameter updates.
ICASSP 2026 [arXiv]
MRIE: Enhanced Rainfall Intensity Estimation with Two-Stage Multimodal Deep Learning
Jin Li, Wentao Hu, Zhigang Zhou.
A two-stage multimodal framework that integrates environmental sensor data and audio to improve rainfall intensity estimation through cross-modal guidance and feature aggregation.
IEEE Access 2025 [doi]
Jin Li, Wentao Hu, Zhigang Zhou.
A two-stage multimodal framework that integrates environmental sensor data and audio to improve rainfall intensity estimation through cross-modal guidance and feature aggregation.
IEEE Access 2025 [doi]
NC-KWS: Few-Shot Class-Incremental Keyword Spotting Based on Neural Collapse
Jin Li, Wenbin Jiang, Yitao Tian, Zhuoyang Li.
Introduces neural-collapse-inspired feature-classifier alignment and an equiangular tight frame classifier for few-shot class-incremental keyword spotting.
NCMMSC 2025 / Springer LNCS [doi]
Jin Li, Wenbin Jiang, Yitao Tian, Zhuoyang Li.
Introduces neural-collapse-inspired feature-classifier alignment and an equiangular tight frame classifier for few-shot class-incremental keyword spotting.
NCMMSC 2025 / Springer LNCS [doi]
- ICASSP 2026 Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
Accepted. [arXiv] - Interspeech 2026 Oral KFC-KWS: Keyframe Fusion with CTC for User-Defined Keyword Spotting
Oral presentation. [arXiv] - NeurIPS 2026 ED Track OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation
Accepted. [arXiv][code] - NCMMSC 2025 NC-KWS: Few-Shot Class-Incremental Keyword Spotting Based on Neural Collapse
[doi] - IEEE Access 2025 MRIE: Enhanced Rainfall Intensity Estimation with Two-Stage Multimodal Deep Learning
[doi]
Projects
HappyTorch
A LeetCode-style, self-hosted PyTorch practice platform.
Provides a focused environment for practicing PyTorch implementation tasks and building intuition for deep learning code.
[code]
A LeetCode-style, self-hosted PyTorch practice platform.
Provides a focused environment for practicing PyTorch implementation tasks and building intuition for deep learning code.
[code]
OpenVTON-Bench
High-resolution benchmark and evaluation protocol for controllable virtual try-on.
Builds a large-scale VTON benchmark with semantic clustering, dense captions, multi-dimensional VLM scoring, and structural metrics for human-aligned evaluation.
[code]
High-resolution benchmark and evaluation protocol for controllable virtual try-on.
Builds a large-scale VTON benchmark with semantic clustering, dense captions, multi-dimensional VLM scoring, and structural metrics for human-aligned evaluation.
[code]
Selected Awards
- Bronze Award, 15th Zhejiang Province "Challenge Cup" College Students' Entrepreneurship Plan Competition, 2026.
- Third Prize, National Undergraduate Physics Experiment Innovation Competition, 2025.
- Third Prize, China College Service Outsourcing Innovation and Entrepreneurship Competition, 2025.
- Zhejiang Provincial Government Scholarship, 2025 & 2024.
- Second Prize, First National ICT Industry-Education Integration Innovation Competition, 2024.
- National Encouragement Scholarship, 2023.
Patents
- Fundus Image Recognition Method, System and Device Based on Deep Learning, CN119444719A, 2025.
- Multimodal Multi-language Custom Instruction Recognition Method and System, CN120690187A, 2025.
- Instruction Recognition Method and System Based on Online Unsupervised Learning, CN118366442A, 2024.