Abstract: One of the most successful applications of LLMs thus far is their use as coding assistants, including tools like Github Copilot or Cursor. However, based on common developer sentiment, there remain many opportunities to improve the productivity and user experience of everyday developers. In this talk, we discuss considerations for designing more effective coding assistants. The first half highlights the gaps in existing evaluations of LLM coding capabilities and proposes a new in-the-wild framework called Copilot Arena. The second half explores proactivity as one avenue for improving interactions with coding assistants. We conclude with a brief discussion of how these considerations can extend to more agentic workflows that are on the horizon for software development.

Abstract: Scientists are experiencing information overload due to the rapid growth of scientific literature. Moreover, the process of discovering new scientific hypotheses has remained slow, expensive, and highly specialist-dependent, due to the increasingly complex experiments. The recent advancements in large language models (LLMs) raise the prospect that they may be able to solve those problems. Despite their impressive progress, these models often fail to incorporate domain-specific knowledge effectively and support their generated results with evidence. To address this issue and lower the entry barrier for interdisciplinary collaboration, I develop AI tools to accelerate and democratize the entire research lifecycle for scientists. I will highlight three stages in the scientific knowledge lifecycle, including (1) the development of scientific knowledge acquisition for limited training data, (2) the integration of domain knowledge in scientific LLM reasoning to narrow search space, and (3) the framework to provide an explainable paper review. Finally, I will outline my future research efforts focused on equipping machines with the ability to interact dynamically with the human and physical world.