Sumit Asthana

Thank you dear visitor for stopping by! I am a final year PhD candidate at University of Michigan, Ann Arbor. My research focus and interests are at the intersection of Machine Learning and Human-Computer Interaction (HCI).

I develop adaptive AI systems that enable people to reason under risk and uncertainty in complex decision-making scenarios by modeling their underlying decision processes and not just their observable behaviors. For example, in education, inferring students’ conceptual gaps requires reconstructing their mental models from their learning trajectories, not just identifying surface-level mistakes. I borrow from cognitive science and probabilistic machine learning to design AI with experts’ mental model to improve Human-AI interaction. By modeling people’s latent cognitive states, my methods improve reasoning of AI systems beyond observed behaviors, improving overall learning efficiency and accuracy. I bring in strong computational and model building skills from my prior industry experience to build systems for Human-AI interaction and my training in HCI allows me to conduct large scale evaluations in people’s work context for improving these systems. For example, I recently built a bayesian network from a massive dataset of 3M Census records to model personal preferences and used it to study the design of personalization agents that respect users’ privacy. I also have strong Reinforcement Learning (RL) foundations that I have applied to model human behavior, which positions me well to explore RL-based fine-tuning of LLMs. For instance, I developed a deep RL system from scratch to simulate indoor human behavior and COVID-19 transmission dynamics, demonstrating how RL can capture and reason about complex behavioral patterns. The following three broad directions describe my research focus and future vision.

Desiging computational models that can understand and improve expert decision-making (AI to critique not obey): Furthering the design of computational models that can understand and reason about experts’ decision processes, and how they reason about and balance principles in their decisions. For example, understanding how instructors balance providing the answer versus guiding students in tutoring scenarios.
Designing Human-AI interaction to support and continuously improve through interaction with experts: Designing user interfaces and interactions that naturally support and augment experts’ work practices, and enable AI to unaimbiguously understand user goals, and mental processes during interactions to improve AI systems during use.
Advancing the responsible use of AI in helping people become experts: Designing methods to advance the responsible use of AI help people become experts by studying decision process of educators and designing technologies that leverage established pedagogical principles to augment educators’s expertise.

I have strong connections with the Wikipedia open-source community. In 2014-18, I extensively contributed to the codebase that powers the mobile version of Wikipedia and was a prominent code reviewer for mobile Wikipedia. I had also administered Wikimedia Foundation’s Google Summer of Code internship program in 2016,17. For my services I was nominated by the Wikimedia Foundation to attend the Google summer of code mentor summit in 2017, and invited to present my research at the monthly Wikimedia research showcase in 2021. Currently, I contribute to the Wikipedia research community, such as serving on the program committee of Wiki Workshop, Wikipedia’s annual research workshop.

selected publications

Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system

Sumit Asthana, Sagih Hilleli, Pengcheng He, and 1 more author

Proc. ACM Hum.-Comput. Interact., Apr 2024

Abs DOI

Meetings play a critical infrastructural role in coordinating work. The recent surge of hybrid and remote meetings in computer-mediated spaces has led to new problems (e.g., more time spent in less engaging meetings) and new opportunities (e.g., automated transcription/captioning and recap support). Advances in dialogue summarization offer the potential for improving post-meeting experiences, but fixed-length summaries often fail to meet diverse needs, such as quick overviews or detailed insights. To address these gaps, we use cognitive science and discourse theories to conceptualize two recap designs: important highlights and a structured, hierarchical minutes view, targeting complementary recap needs. We operationalize these representations into high-fidelity prototypes using dialogue summarization. Finally, we evaluate the representations’ effectiveness with seven users in the context of their work meetings at Microsoft. Our results show both recap types are valuable in different contexts, enabling collaboration through discussions and consensus-building. Exploring the meaning of users adding, editing, and deleting from recaps suggests varying alignment for using these actions to improve AI-recap. Our design implications, such as incorporating organizational artifacts (e.g., linking presentations) in recaps and personalizing context, advance the discourse of effective recap designs for organizational work and support past results from cognition studies.
"I know even if you don’t tell me": Understanding Users’ Privacy Preferences Regarding AI-based Inferences of Sensitive Information for Personalization

Sumit Asthana, Jane Im, Zhe Chen, and 1 more author

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, Apr 2024

Abs DOI

Personalization improves user experience by tailoring interactions relevant to each user’s background and preferences. However, personalization requires information about users that platforms often collect without their awareness or their enthusiastic consent. Here, we study how the transparency of AI inferences on users’ personal data affects their privacy decisions and sentiments when sharing data for personalization. We conducted two experiments where participants (N=877) answered questions about themselves for personalized public arts recommendations. Participants indicated their consent to let the system use their inferred data and explicitly provided data after awareness of inferences. Our results show that participants chose restrictive consent decisions for sensitive and incorrect inferences about them and for their answers that led to such inferences. Our findings expand existing privacy discourse to inferences and inform future directions for shaping existing consent mechanisms in light of increasingly pervasive AI inferences.
Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts

Sumit Asthana, Hannah Rashkin, Elizabeth Clark, and 2 more authors

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

Abs DOI

One useful application of NLP models is to support people in reading complex text from unfamiliar domains (e.g., scientific articles). Simplifying the entire text makes it understandable but sometimes removes important details. On the contrary, helping adult readers understand difficult concepts in context can enhance their vocabulary and knowledge. In a preliminary human study, we first identify that lack of context and unfamiliarity with difficult concepts is a major reason for adult readers’ difficulty with domain-specific text. We then introduce targeted concept simplification, a simplification task for rewriting text to help readers comprehend text containing unfamiliar concepts. We also introduce WikiDomains, a new dataset of 22k definitions from 13 academic domains paired with a difficult concept within each definition. We benchmark the performance of open-source and commercial LLMs and a simple dictionary baseline on this task across human judgments of ease of understanding and meaning preservation. Interestingly, our human judges preferred explanations about the difficult concept more than simplifications of the concept phrase. Further, no single model achieved superior performance across all quality dimensions, and automated metrics also show low correlations with human evaluations of concept simplification (~0.2), opening up rich avenues for research on personalized human reading comprehension support.