Perception and LANguage (PLAN) Group
Our Perception and LANguage (PLAN) research lab is broadly interested in multimodal machine learning and learning with limited supervision. We are particularly interested in building intelligent task assistants that can fuse linguistic, visual, and other types of modalities to perceive and interact with the world. Current language + vision projects involve multimodal representation learning, contrastive self-supervision, embodied AI, video localization and multi-agent communication. Applications include healthcare, medical imaging, manufacturing and misinformation detection.
News
- June 2025: Congratulations to Kajal for being recognized as Outstanding 2024-2025 SPIN Intern!
- March 2025: Purpcorn-PLAN team is competing in the Amazon Nova AI Challenge!
- February 2025: Congratulations to Kiet, Adheesh, Joey, and Muntasir, and for their CVPR'25 paper on part-focused semantic co-segmentation with VLMs.
- February 2025: Our lab has received DARPA funding.
- February 2025: Our lab has received DoD funding.
- August 2024: PLAN Lab is moving to the University of Illinois at Urbana-Champaign.
- December 2023: Congratulations to Meghana for her AAAI'24 paper on zero-shot language video localization.
- December 2023: Congratulations to Hulya and Kiet for their CSCW'24 paper on examining online U.S. political discourse related to refugees.
- December 2023: Congratulations to Afrina for her ICASSP'24 paper on finetuning audio Transformers with unsupervised audio mixtures (internship work @ Microsoft Research).
- December 2023: Congratulations to Muntasir, Xiaona, and Joey for their WACV'24 paper on fine-grained alignment for cross-modal recipe retrieval.
- October 2023: Congratulations to Ying for receiving an Amazon-VT Fellowship!
- October 2023: PLAN Lab has received new Amazon-VT funding to continue work on Embodied AI.
- September 2023: 🏆 Congratulations to the VT PLANBot team for advancing to the AlexaPrize TaskBot Challenge 2 finals! 🚀 Go Hokies!
- August 2023: Our lab has received DARPA funding.
- August 2023: Congratulations to Muntasir for receiving a VT Pratt Fellowship for the Fall 2023 semester and first-authoring a CIKM 2023 Applied Research accepted paper.
- August 2023: Congratulations to Amarachi for her first-authored MICCAI 2023 work on ViTs for disease progression.
- July 2023: Congratulations to the PLANBot team for advancing to the AlexaPrize TaskBot Challenge 2 semifinals!
- June 2023: 🎓 Congratulations to Ming for successfully defending her Ph.D. thesis!
- May 2023: 🎓 Congratulations to Avi, Gaurang, Hulya, and Meghana for graduating!
- May 2023: Congratulations to Vedant Shah for receiving a David Heilman Award for Excellence in Undergraduate Research!
- April 2023: Congratulations to Meghana Holla for being selected as the Master's Paul E. Torgersen Research Excellence Award winner!
- April 2023: Makanjuola is featured at VT Daily News as one of the eight recipients of the Cadence Black Students in Technology Scholarship!
- April 2023: Thank you to the Virginia Tech Student Engineers' Council (SEC) for the Big Contribution grant.
- Feb. 2023: Makanjuola is being featured in a spotlight article by Cadence with other recipients of the Cadence Black Students in Technology Scholarship. Congratulations to all recipients!
- Feb. 2023: Congratulations to our PLANBot team (Afrina, Muntasir, Makanjuola, Amarachi, and Joey) for being selected for the AlexaPrize TaskBot Challenge 2!
- Feb. 2023: Congratulations to Joey for his paper acceptance at TheWebConf2023 conference.
- Feb. 2023: Congratulations to Amarachi for her co-authored PLOS Digital Health editorial on ChatGPT and medical education.
- Nov. 2022: Congratulations to Makanjuola for receiving a 2022 Cadence Diversity in Technology Scholarship!
- Nov. 2022: Congratulations to Muntasir for receiving a VT Pratt Fellowship for the Spring 2023 semester!
- Oct. 2022: Congratulations to Muntasir and Avi for their paper acceptance at the 2022 IEEE BigData conference.
- Oct. 2022: Congratulations to Hesam Soleimani for his journal accepted at the Mechanical Systems and Signal Processing (MSSP).
- Oct. 2022: Our lab has received an Amazon-VT faculty research award to work on Embodied AI research.
- Sept. 2022: Congratulations to Gaurang and Amarachi for the MICCAI 2022 paper acceptance.
- August 2022: Our lab has received DARPA funding.
- July 2022: Our lab has received Commonwealth Cyber Initiative (CCI) funding.
- February 2022: Our lab has received NSF EAGER CMMI-AM funding.