Yin Li
Assistant Professor
Biostatistics & Medical Informatics
Computer Sciences
University of Wisconsin-Madison
6730, Medical Science Center
yin.li at wisc.edu
Short Bio Google Scholar Profile
RESEARCH
Computer Vision, Mobile Health, Machine Learning
TEAM
Abrar Majeedi (BDS Ph.D. student)
Zihao Zhu (CS Ph.D. student)
Zhuoyan Xu (Stats Ph.D. student, co-supervised with Prof. Yingyu Liang)
Zhuoming Liu (CS Ph.D. student)
Alex Huang (CS undergrad student, Hilldale Fellow 2024)
Yiquan Li (CS undergrad student)
To work with me:
Students: See my note.
Postdocs: I am not looking for postdocs at this moment.
ALUMNI
[Those who spend >= 2 semesters with us]
Satya Sai Srinath Namburi GNVV (MS CS, 2022-2024). Now SDE at GE HealthCare
Viswanatha Reddy Gajjala (MS BDS / CS, 2021-2023). Now SDE at Amazon
Zhengyang Lou (PhD ECE, co-supervised with Prof. Yu Hen Hu, 2020-2024). Now Senior Algorithm Engineer at NIO
Fangzhou Mu (PhD CS, 2018-2023). Now Senior Machine Learning Algorithm Engineer at Nvidia
Yiwu Zhong (PhD CS, 2018-2023). Now postdoc at CUHK
Evelin Yin (BS CS, 2022-2023). Now CS PhD student at Purdue
Sicheng Mo (BS CS, 2022-2022). Now CS MS student at UCLA
Cameron Ruggles (CS MS, 2021-2023). TBD
Liancheng Fang (BDS MS, 2020-2021). Now SDE at United Sensing Technology
Siddeshwar Raghavan (MS ECE, 2019-2021). Now ECE PhD student at Purdue
Zitong Zhan (BS CS, 2020-2021). Now CS MS student at UIUC
Abrar Majeedi (MS BDS, 2020-2021). Now BDS PhD student at UW Madison
Akash Sharma (MS CS, 2020). Now SDE at Microsoft
Chen-Lin Zhang (2019-2020).
Visiting Ph.D. student from Nanjing University. Now Algorithm Engineer at 4Paradigm
Zixuan Huang (MS CS, 2018-2020).
Now CS PhD student at Georgia Tech
Zhe Huang (BS CS, 2019-2020).
Now MS Robotics student at CMU
Shashank Verma (MS ECE, 2019-2020). Now SDE at Nvidia
Yuke Liang (BS CS, 2019). Now SDE at Twitter
TEACHING
Fall 2024:
BMI 771 / CS 771
Learning Based Methods for Computer Vision
Fall 2023:
BMI 771 / CS 771
Learning Based Methods for Computer Vision
Fall 2022:
BMI 771 / CS 771
Learning Based Methods for Computer Vision
Fall 2021:
BMI 826
Learning Based Methods in Computer Vision
Fall 2020:
CS540
Introduction to Artificial Intelligence
Fall 2019:
BMI 826 / CS 838
Learning Based Methods in Computer Vision
Spring 2019:
BMI 826 / CS 838
Learning Based Methods in Computer Vision
NEWS
Sep 2024: One paper (freehand 3D ultrasound reconstruction) accepted to IEEE TUFFC.
Sep 2024: I am serving as an area chair for CVPR 2025.
July 2024: One paper accepted to ECCV.
June 2024: I am serving as an area chair for WACV 2025.
June 2024: One paper (quantification of care manipulation in NICU with video and sensor data) accepted to npj Digital Medicine.
May 2024: I am serving as an area chair for NeurIPS 2024.
Feb 2024: Three papers accepted to CVPR.
Jan 2024: One paper accepted to ICLR.
Dec 2023: One paper (drone-based object tracking benchmark) accepted to IJCV. See more details here.
Nov 2023: I am serving as an area chair for ECCV 2024.
July 2023: Two papers accepted to ICCV.
June 2023: I am serving as an area chair for CVPR 2024.
June 2023: Our team was the first place winner in the 2023 challenges of EPIC-Kitchens for Action Detection and the second place winner in the 2023 Ego4D Challenge for the Episodic memory: Moments queries (MQ).
June 2023: One paper (automatic segmentation of recylcing materials in contruction sites) accepted to Automation in Construction.
Mar 2023: With MGH (Harvard), NCI, Sage Bionetworks, and Intel, we will be hosting the first neurofibromatosis tumor segmentation on whole-body MRI challenge (WBMRI-NF). See more details here.
Mar 2023: I am serving as an area chair for NeurIPS 2023.
Feb 2023: One paper accepted to CVPR.
Jan 2023: One paper accepted to ICLR.
Dec 2022: Papers accepted to TODAES and Pattern Recognition.
Oct 2022: Our team was the second place winners in the 2022 Ego4D Challenge for Episodic Memory Moments and Natural Language Queries (MQ and NLQ).
Sep 2022: One paper accepted to NeurIPS Datasets and Benchmarks track. See the project page here.
Sep 2022: I am serving as an area chair for CVPR 2023.
Aug 2022: Our team was the first place winner in the VIMS/IAARC Datathon 2022 competition (link to certificate).
Aug 2022: One paper accepted to WACV (round 1).
July 2022: Four papers accepted to ECCV.
June 2022: Two papers accepted to ICCP, including one conditionally accepted to TPAMI Special Issue on Computational Photography.
May 2022: I am serving as an area chair for WACV 2023.
Apr 2022: We are hosting UW Madison GI Tract Image Segmentation challenge on Kaggle.
Mar 2022: Three papers accepted to CVPR (one oral and two posters).
Feb 2022: Check out our new tech report with code on temporal action localization, with SOTA performance across benchmarks. Update: the paper is accepted to ECCV.
Feb 2022: I am serving as an area chair for ECCV 2022.
Jan 2022: Our paper on the design of adaptive video object detection systems (with Purdue) was accepted to EuroSys.
Dec 2021: Check out our new project on creating stylized 3D photos from a single image. Update: The paper is accepted to CVPR 2022 as an oral presentation.
July 2021: Two papers accepted to ICCV (two posters).
June 2021: Our work (with Purdue) on benchmarking object detectors on embedded devices was presented at EMDL workshop, MobiSys 2021. See the project page here
June 2021: I am serving as an area chair for WACV 2022.
Mar 2021: Two papers accepted to CVPR (one oral and one poster).
Jan 2021: One paper accepted to IEEE TPAMI (arXiv preprint).
Dec 2020: One paper accepted to AAAI.
Oct 2020: One paper with Purdue accepted to SenSys.
Oct 2020: Our work on lifting load prediction was presented at HFES 2020. See the project page here.
Sep 2020: Our presentation of FingerTrak was selected as a best presentation nominee at UbiComp 2020.
Aug 2020: Our demo on wearable 3D hand tracking (FingerTrak) was selected as a best demo nominee (3/41) at ECCV 2020.
Aug 2020: Our work on wearable hand tracking (with Cornell) was recently coverd by UW Madison News and Cornell Chronicle, and featured in Engadget, Gizmodo, VentureBeat, BBC, Yahoo News and Forbes. We will also present a demo at ECCV 2020 and our paper at UbiComp 2020.
Aug 2020: One paper accepted to BMVC as an oral presentation.
July 2020: I am serving as an area chair for IJCAI 2021.
July 2020: Two papers accepted to ECCV (one oral and one poster).
June 2020: Congratulations to Miao Liu for winning EPIC-Kitchens Challenge 2020 (3rd place in action recognition). See our preprint here.
May 2020: One paper with Cornell accepted to IMWUT (UbiComp 2020).
Mar 2020: One paper accepted to CVPR as an oral presentation.
Dec 2019: One paper accepted to ICLR.
Oct 2019: I am serving as an area chair for ECCV 2020.
Jun 2019: I am serving as an area chair for WACV 2020.
Jan 2019: One paper accepted to IEEE TIP.
Nov 2018: I am serving as area chairs for CVPR'19 and ICCV'19.
Oct 2018: One paper accepted to IEEE TIP.
Sep 2018: One paper accepted to NeurIPS.
Aug 2018: Joined UW-Madison as an assistant professor.
HIGHLIGHTS
ActionFormer: Localizing Moments of Actions with Transformers
3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
RegionCLIP: Region-based Language-Image Pretraining
Learning to Generate Scene Graph from Natural Language Supervision
DSMIL for Whole Slide Image Classification
Comprehensive Image Captioning via Scene Graph Decomposition
Interpretable and Accurate Fine-grained Recognition via Region Grouping
Gradients as Features for Deep Representation Learning
Learning to Grasp without Seeing
Georgia Tech First Person Vision Repository
Unsupervised Learning of Edges from Video
BioGlass
MIT Tech Review,
New Scientist,
WIRED UK
Detecting Bids for Eye Contact using a Wearable Camera
PASCAL-S Dataset
(The Secrets of Salient Object Segmentation)
SELECTED PUBLICATIONS
ECCV 2024:
Abrar Majeedi, Viswanatha Reddy Gajjala, Satya Sai Srinath Namburi GNVV, Yin Li
RICA2: Rubric-Informed, Calibrated Assessment of Actions
[project]
[arXiv]
npj Digital Medicine 2024:
Abrar Majeedi, Ryan M. McAdams, Ravneet Kaur, Shubham Gupta, Harpreet Singh, Yin Li
Deep Learning to Quantify Care Manipulation Activities in Neonatal Intensive Care Units
[paper]
[project/code]
CVPR 2024:
Fangzhou Mu*, Carter Sifferman*, Sacha Jungerman, Yiquan Li, Zhiyue Han, Michael Gleicher, Mohit Gupta, Yin Li (* equal contribution)
Towards 3D Vision with Low-Cost Single-Photon Cameras
[project]
[arXiv]
CVPR 2024:
Fangzhou Mu*, Sicheng Mo*, Yin Li (* equal contribution)
SnAG: Scalable and Accurate Video Grounding
[project/code]
[arXiv]
CVPR 2024:
Sicheng Mo*, Fangzhou Mu*, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, Bolei Zhou (* equal contribution)
FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition
[project]
[code]
[arXiv]
ICLR 2024:
Zhuoyan Xu, Zhenmei Shi, Junyi Wei, Fangzhou Mu, Yin Li, Yingyu Liang
Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning
[project/code]
[paper]
IJCV 2023:
Xin Zhao, Shiyu Hu, Yipei Wang, Jing Zhang, Yimin Hu, Rongshuai Liu, Haibin Ling, Yin Li, Renshu Li, Kun Liu, Jiadong Li
BioDrone: A Bionic Drone-Based Single Object Tracking Benchmark for Robust Vision
[project/dataset]
[paper link]
[arXiv]
ICCV 2023:
Felipe Gutierrez-Barragan, Fangzhou Mu, Andrei Ardelean, Atul Ingle, Claudio Bruschini, Edoardo Charbon, Yin Li, Mohit Gupta, Andreas Velten
Learned Compressive Representations for Single-Photon 3D Imaging
[pdf]
ICCV 2023:
Matthew Dutson, Yin Li, Mohit Gupta
Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers
[project]
[pdf]
[code]
CVPR 2023:
Yiwu Zhong, Licheng Yu, Yang Bai, Shangwen Li, Xueting Yan*, Yin Li* (* co-corresponding authors)
Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations
[arXiv]
[pdf]
[code]
ICLR 2023:
Zhuoran Yu, Yin Li, Yong Jae Lee
InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learning
[link]
[code]
NeurIPS 2022 (Datasets and Benchmarks Track):
Sizhe An, Yin Li, Umit Y. Ogras
mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors
[link]
[project/code]
ECCV 2022:
Chen-Lin Zhang, Jianxin Wu, Yin Li
ActionFormer: Localizing Moments of Actions with Transformers
[arXiv]
[pdf]
[project/code]
ECCV 2022:
Matthew Dutson, Yin Li, Mohit Gupta
Event Neural Networks
[arXiv]
[pdf]
[project/code]
ECCV 2022:
Sacha Jungerman, Atul Ingle, Yin Li, Mohit Gupta
3D Scene Inference from Transient Histograms
[pdf]
[project]
ECCV 2022:
Miao Liu, Lingni Ma, Kiran Somasundaram, Yin Li, Kristen Grauman, James Rehg, Chao Li
Egocentric Activity Recognition and Localization on a 3D Map
[arXiv]
[pdf]
ICCP/TPAMI 2022:
Fangzhou Mu, Sicheng Mo, Jiayong Peng, Xiaochun Liu, Ji Hyun Nam, Siddeshwar Raghavan, Andreas Velten, Yin Li
Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging
[arXiv]
[link]
[project]
[code]
ICCP 2022:
Bhavya Goyal, Jean-François Lalonde, Yin Li, Mohit Gupta
Robust Scene Inference under Noise-Blur Dual Corruptions
[arXiv]
[project]
CVPR 2022:
Ran Xu, Fangzhou Mu, Jayoung Lee, Preeti Mukherjee, Somali Chaterji, Saurabh Bagchi, Yin Li
SMARTADAPT: Multi-branch Object Detection Framework for Videos on Mobiles
[pdf]
[project]
CVPR 2022:
Fangzhou Mu, Jian Wang*, Yicheng Wu*, Yin Li* (* co-corresponding authors)
3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
[pdf]
[project]
[code]
(Oral, acceptance rate 4.2%)
CVPR 2022:
Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, Jianfeng Gao
RegionCLIP: Region-based Language-Image Pretraining
[arXiv]
[pdf]
[project/code/demo]
EuroSys 2022:
Ran Xu, Jayoung Lee, Pengcheng Wang, Saurabh Bagchi, Yin Li, Somali Chaterji
LiteReconfig: Cost and Content Aware Reconfiguration of Video Object Detection Systems for Mobile GPUs
[link]
[code]
ICCV 2021:
Yiwu Zhong, Jing Shi, Jianwei Yang, Chenliang Xu, Yin Li
Learning to Generate Scene Graph from Natural Language Supervision
[arXiv]
[pdf]
[project/code]
ICCV 2021:
Jing Shi, Yiwu Zhong, Ning Xu, Yin Li, Chenliang Xu
A Simple Baseline for Weakly-Supervised Scene Graph Generation
[pdf]
CVPR 2021:
Bin Li, Yin Li*, Kevin W. Eliceiri* (* co-corresponding authors)
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning
[pdf]
[code]
(Oral, acceptance rate 4.0%)
CVPR 2021:
Liwei Wang, Jing Huang, Yin Li, Kun Xu, Zhengyuan Yang, Dong Yu
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
[pdf]
[code]
TPAMI 2021:
Yin Li, Miao Liu, James M. Rehg
In the Eye of the Beholder: Gaze and Actions in First Person Video
[arXiv]
[link]
AAAI 2021:
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, Vikas Singh
Nyströmformer: A Nyström-based Algorithm for Approximating Self-Attention
[pdf]
[project/code]
SenSys 2020:
Ran Xu, Chen-Lin Zhang, Pengcheng Wang, Jayoung Lee, Subrata Mitra, Somali Chaterji, Yin Li, Saurabh Bagchi
ApproxDet: Content and Contention-Aware Approximate Object Detection for Mobiles
[arXiv]
[link]
[project]
[code]
BMVC 2020:
Miao Liu, Xin Chen, Yun Zhang, Yin Li, James M. Rehg
Attention Distillation for Learning Video Representations
[arXiv]
[pdf]
[project]
(Oral, acceptance rate 5.0%)
ECCV 2020:
Yiwu Zhong, Liwei Wang, Jianshu Chen, Dong Yu, Yin Li
Comprehensive Image Captioning via Scene Graph Decomposition
[arXiv]
[pdf]
[project]
[code]
ECCV 2020:
Miao Liu, Siyu Tang, Yin Li, James M. Rehg
Forecasting Human Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Vision
[arXiv]
[pdf]
[project]
[code]
(Oral, acceptance rate 2.0%)
IMUWT/UbiComp 2020:
Fang Hu, Peng He, Songlin Xu, Yin Li, Cheng Zhang
FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist
[link]
[project]
[video demo]
(Best Demo Nominee at ECCV 2020, 3 out of 41)
CVPR 2020:
Zixuan Huang, Yin Li
Interpretable and Accurate Fine-grained Recognition via Region Grouping
[arXiv]
[project]
[code]
(Oral, acceptance rate 5.7%)
ICLR 2020:
Fangzhou Mu, Yingyu Liang, Yin Li
Gradients as Features for Deep Representation Learning
[OpenReview]
[project]
[code]
NeurIPS 2018:
Yin Li, Abhinav Gupta
Beyond Grids: Learning Graph Representations for Visual Recognition
[pdf]
ECCV 2018:
Yin Li, Miao Liu, James M. Rehg
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Vision
[pdf]
ECCV 2018:
Keizo Kato, Yin Li, Abhinav Gupta
Compositional Learning of Human Object Interactions
[pdf]
ISER 2018:
Adithyavairavan Murali, Yin Li, Dhiraj Gandhi, Abhinav Gupta
Learning to Grasp without Seeing
[arXiv]
[project]
CVPR 2018:
Abhijit Kundu, Yin Li, James M. Rehg
3D-RCNN: Instance-level 3D Object Reconstruction via Render-and-Compare
[pdf]
[project]
(Oral, acceptance rate 2.1%)
TPAMI 2018:
Liwei Wang, Yin Li, Jing Huang, Svetlana Lazebnik
Learning Two-branch Neural Networks for Image-text Matching Tasks
[arXiv]
[link]
TCYB 2018:
Junchi Yan, Changsheng Li, Yin Li, Guitao Cao
Adaptive Discrete Hypergraph Matching
[link]
JADD 2017:
Sarah R. Edmunds, Agata Rozga, Yin Li, Elizabeth A. Karp, Lisa V. Ibanez, James M. Rehg, Wendy L. Stone
Using a Point-of-View Camera to Measure Eye Gaze in Young Children with Autism Spectrum Disorder During Naturalistic Social Interactions: A Pilot Study
[link]
CVPR 2016:
Yin Li, Manohar Paluri, James M. Rehg, Piotr Dollár
Unsupervised Learning of Edges
[arXiv]
[project]
(Oral, acceptance rate 3.9%)
CVPR 2016:
Liwei Wang, Yin Li, Svetlana Lazebnik
Learning Deep Structure-Preserving Image-Text Embeddings
[arXiv]
[code]
[data splits]
CVPR 2015:
Yin Li, Zhefan Ye, James M. Rehg
Delving into Egocentric Actions
[pdf]
[dataset]
CVPR 2015:
Jia Xu, Lopamudra Mukherjee, Yin Li, Jamieson Warner, James M. Rehg, Vikas Singh
Gaze-enabled Egocentric Video Summarization via Constrained Submodular Maximization
[pdf]
[project]
FG 2015:
Zhefan Ye, Yin Li, Yun Liu, Chanel Bridge, Agata Rozga, James M. Rehg
Detecting Bids for Eye Contact using a Wearable Camera
[pdf]
[project]
(Best Student Paper Award, 1/84)
MobiHealth 2014:
Javier Hernandez, Yin Li, James M. Rehg, Rosalind W. Picard
BioGlass: Physiological Parameter Estimation Using a Head-mounted Wearable Device
[pdf]
[project]
(Best Student Paper Award, 1/120)
ECCV 2014:
Junchi Yan, Yin Li, Wei Liu, Hongyuan Zha, Xiaokang Yang
Graduated Consistency-Regularized Optimization for Multi-Graph Matching
[pdf]
ECCV 2014:
Abhijit Kundu, Yin Li, Frank Dellaert, Fuxin Li, James M. Rehg
Joint Semantic Segmentation and 3D Reconstruction from Monocular Video
[pdf]
[project]
CVPR 2014:
Yin Li*, Xiaodi Hou*, Christof Koch, James M. Rehg, Alan L. Yuille
The Secrets of Salient Object Segmentation
[arXiv]
[project]
(* denotes equal contribution)
ICCV 2013:
Yin Li, Alireza Fathi, James M. Rehg
Learning to Predict Gaze in Egocentric Video
[pdf]
[dataset]
(Oral, acceptance rate 2.5%)
CVPR 2013:
James M. Rehg et al.
Decoding Children's Social Behavior
[pdf]
[dataset]
ECCV 2012:
Alireza Fathi, Yin Li, James M. Rehg
Learning to Recognize Daily Actions using Gaze
[pdf]
[dataset]
CVPR 2012:
Liwei Wang, Yin Li, Jiaya Jia, Jian Sun, David P. Wipf, James M. Rehg
Learning Sparse Covariance Patterns for Natural Scenes
[pdf]
[code]
ECCV 2010:
Yin Li, Junchi Yan, Yue Zhou and Jie Yang
Optimum Subspace Learning and Error Correction for Tensors
[link]