We aim to develop a Responsible AI (RAI) assessment framework consisting of evaluation metrics and mitigation strategies. This research aims to combine industry and academic perspectives to create a comprehensive guide for building RAI. As AI technology advances, governments are introducing requirements for transparency, accountability, and data protection. Compliance with these regulations is essential to avoid substantial fines and penalties. The absence of clear metrics and key performance indicators (KPIs) poses a challenge to assessing RAI, but companies can develop and apply relevant metrics within a robust framework. Regular assessments offer valuable insights into AI systems' current states, highlighting areas of excellence and needed improvements, allowing for targeted actions to enhance AI governance. Identifying RAI weaknesses can also help secure funding for necessary initiatives. Mitigating risks such as bias, discrimination, and privacy violations is paramount, and rigorous RAI assessments enable proactive risk management, reducing potential harm to individuals and society. The proposed RAI assessment supports regulatory compliance, risk mitigation, and continuous improvement in AI governance and ethical responsibility.

AI practitioners often need to perform model comparison to select the most suitable models for integration into larger pipelines, or to cross-check fine-tuned models against their predecessors for desired improvements. Our goal is to enhance the current model comparison pipeline in computer vision by eliminating the need for exhaustive image annotation to collect validation sets, and by providing more informative and actionable insights than conventional accuracy metrics. We are developing a tool that leverages model disagreement signals as a search mechanism on large, unlabeled data sources, thereby bypassing the need for human annotations. Our tool identifies and produces "Error Pockets"—thematic groups of images where two models disagree, which can be coherently described by humans. These error pockets allow practitioners to examine the systematic strengths and weaknesses of any model in the context of their specific application. This approach provides a deeper audit beyond the simplistic analysis afforded by numerical metrics and offers prescriptive data for model improvement with human-understandable explanations. By incorporating sparse feedback from practitioners, our tool aligns error pocket searches with human preferences. Our experiments demonstrate that training models on their identified error pockets significantly improves performance on those specific error modes. We are currently validating the effectiveness, understandability, and actionability of error pockets through user studies with real AI practitioners who have various model comparison needs.

Deep reinforcement learning (DRL) is a powerful machine learning paradigm for generating agents that control autonomous systems. However, the black-box nature of DRL agents limits their deployment in real-world safety-critical applications. A promising approach for providing strong guarantees on an agent's behavior is to use Neural Lyapunov Barrier (NLB) certificates, which are learned functions over the system whose properties indirectly imply that an agent behaves as desired. In this project, we explore the joint training of neural controllers and NLB certificates. We use a formal neural network verification tool (Marabou) to check the certificate and provide counterexample-guided feedback to the training. We also explore techniques for improving scalability and compositionality and showcase our techniques on autonomous vehicle case studies.

Recently, 3D neural maps learned from sensor data have shown astounding results in visual realism and detail, as well as more compactness and efficiency compared to traditional map representations. However, despite their strengths over traditional map representations, neural maps are still limited by incompletely observable and highly dynamic environments, rendering their use in safety-critical systems challenging. We aim to leverage advancements in neural scene reconstruction to develop high-quality neural mapping for environments ranging from dense cities to extraterrestrial terrain. We are working to develop safe methods for downstream navigation applications (localization, error forecasting, and path-planning) that are robust to diverse sources of map errors. Additionally, we are working on methods to quantify and mitigate uncertainty in map regions, provide bounds on localization error, and ensure safety guarantees for trajectory planning.