Publications
Research papers from the Mi3 Lab
For a complete list, see Google Scholar.
2025
Can Vision-Language Models Understand and Interpret Dynamic Gestures from Pedestrians? Pilot Datasets and Exploration Towards Instructive Nonverbal Commands for Cooperative Autonomous Vehicles
CVPR Workshop on Foundation Models for V2X-Based Cooperative Autonomous Driving (DriveX)
Best Application Paper AwardEvaluating Multimodal Vision-Language Model Prompting Strategies for Visual Question Answering in Road Scene Understanding
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), LLVM-AD Workshop
Bosch Best Paper AwardAutomated Context-Aware Navigation Support for Individuals with Visual Impairment Using Multimodal Language Models in Urban Environments
CVPR Workshop on Accessibility, Vision, and Autonomtic (AVA)
1st Place, Navigation Instruction Generation ChallengeLanguage-Driven Active Learning for Diverse Open-Set 3D Object Detection
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Towards a Multi-Agent Vision-Language System for Zero-Shot Novel Hazardous Object Detection for Autonomous Driving Safety
arXiv preprint
Lights as Points: Learning to Look at Vehicle Substructures with Anchor-Free Object Detection
IEEE Robotics and Automation Letters (RA-L)
Improving Event-Phase Captions in Multi-View Urban Traffic Videos via Prompt-Aware LoRA Tuning of Vision Language Models
ICCV Workshop on ROAD Challenge
Automated Data Curation Using GPS & NLP to Generate Instruction-Action Pairs for Autonomous Vehicle Vision-Language Navigation Datasets
arXiv preprint
MTR-VP: Towards End-to-End Trajectory Planning through Context-Driven Image Encoding and Multiple Trajectory Prediction
arXiv preprint
DepthVision: Robust Vision-Language Understanding through GAN-Based LiDAR-to-RGB Synthesis
arXiv preprint
ImproVision: Visual Communication and Human-Computer Interactions for Musical Creativity
Leonardo (MIT Press)
ImproVision Equilibrium: Toward Multimodal Musical Human-Machine Interaction
Transactions of the International Society for Music Information Retrieval (TISMIR)
Towards Vision Zero: The TUM Traffic Accid3nD Dataset
IEEE/CVF International Conference on Computer Vision (ICCV)
A New Perspective On AI Safety Through Control Theory Methodologies
IEEE Open Journal of Intelligent Transportation Systems
2024
doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation
arXiv preprint
Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving
arXiv preprint
Pedestrian Safety by Intent Prediction: A Lightweight LSTM-Attention Architecture and Experimental Evaluations with Real-World Datasets
IEEE Intelligent Vehicles Symposium (IV)
Patterns of Vehicle Lights: Addressing Complexities of Camera-Based Vehicle Light Datasets and Metrics
Pattern Recognition Letters
ActiveAnno3D: An Active Learning Framework for Multi-Modal 3D Object Detection
IEEE Intelligent Vehicles Symposium (IV)
The Why, When, and How to Use Active Learning in Large-Data-Driven 3D Object Detection for Safe Autonomous Driving
arXiv preprint
Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning
arXiv preprint
Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection
European Conference on Computer Vision (ECCV)
Creativity and Visual Communication from Machine to Musician: Sharing a Score through a Robotic Camera
International Conference on ArtsIT, Interactivity and Game Creation
Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets
IEEE Vehicular Technology Conference (VTC-Fall)
Learning to Find Missing Video Frames with Synthetic Data Augmentation: A General Framework and Application in Generating Thermal Images Using RGB Cameras
IEEE Intelligent Vehicles Symposium (IV)
Towards Safe, Human-Centered Autonomous Driving: Real-World Artificial Intelligence for Enhanced Situation Awareness and Transition Control
University of California, San Diego (PhD Dissertation)
2023
Robust Traffic Light Detection Using Salience-Sensitive Loss: Computational Framework and Evaluations
IEEE Intelligent Vehicles Symposium (IV)
Robust Detection, Association, and Localization of Vehicle Lights: A Context-Based Cascaded CNN Approach and Evaluations
arXiv preprint
Salient Sign Detection in Safe Autonomous Driving: AI Which Reasons Over Full Visual Context
International Technical Conference on the Enhanced Safety of Vehicles (ESV)
Safe Control Transitions: Machine Vision Based Observable Readiness Index and Data-Driven Takeover Time Prediction
International Technical Conference on the Enhanced Safety of Vehicles (ESV)
Pedestrian Behavior Maps for Safety Advisories: CHAMP Framework and Real-World Data Analysis
IEEE Intelligent Vehicles Symposium (IV)
Ensemble Learning for Fusion of Multiview Vision with Occlusion and Missing Information
arXiv preprint
Spectrogram-Based Deep Learning for Flute Audition Assessment and Intelligent Feedback
IEEE International Symposium on Multimedia (ISM)
Deep and Shallow: Machine Learning in Music and Audio
Chapman and Hall/CRC (Textbook)
2022
On Salience-Sensitive Sign Classification in Autonomous Vehicle Path Planning: Experimental Explorations with a Novel Dataset
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
From Pedestrian Detection to Crosswalk Estimation: An EM Algorithm, Analysis, and Evaluations on Diverse Datasets
ICML Workshop on Safe Learning for Autonomous Driving
2021
Trajectory Prediction in Autonomous Driving with a Lane Heading Auxiliary Loss
IEEE Robotics and Automation Letters (RA-L)
Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data
IEEE International Intelligent Transportation Systems Conference (ITSC)
Predicting Take-Over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation
arXiv preprint
Restoring Eye Contact to the Virtual Classroom with Machine Learning
International Conference on Computer Supported Education (CSEDU)