I’m doing it with LLMs or I’m not doing it at all.

📜 google scholar | ✉️ aholtzman@uchicago.edu |

Head of Conceptualization Lab
Assistant Professor at

Office: Searle 213
I am actively looking for PhD students to apply through both UChicago CS and the Data Science Institute. Experience with LLMs/generative models is a plus, but not required! I no longer have time to reply to all emails from prospective PhD students but I highly encourage you to check out my lab website and apply!
Previously:

Post-Doc @ FAIR @ Meta
PhD from UW advised by Luke Zettlemoyer

Natural Language Generation
Machine Communication
Generative Models as a Complex Systems Science

Holtzman CV.pdf110.5KB

Events

2025/06—excited to give a talk at MMLS!
2024/10—The UChicago Communication and Intelligence Symposium was a great success!
2024/07—Officially started at UChicago as an Assistant Professor!
2023/09—Started a Post-doc at Meta
2023/09—Gave a keynote talk at the International Conference on Social Computing about LLMs as Linguistic Spies

Nightly Research Statement

Large language models farm corpora for the heuristics and strategies embedded in data. How can we extract the abstractions crystalized in LLMs?
What can we do with LLMs that has never been done, or even thought of, before? Can we pushback against the current bounds of ineffability to help humans articulate what was previously inarticulable? Can we make narrative video games with a dense graph of states where one learns through bad decisions instead of replaying failures?

Research Foci

My primary interest is in generative models, how they work and how we can get them to generate text and other media that communicate with humans is useful and novel ways. Lately, I’ve been thinking about how language models fit the definition of complex systems, systems in which we understand the low-level components (neurons) but can’t explain or even fully describe the high-level behaviors (e.g., in-context learning) as they emerge with more data and parameters. In the spirit of complex systems, I want to create a taxonomy of model behavior, analogous to the periodic table of elements in Chemistry, which hardly explains complex chemical processes in its own right, but gives a description of elementary components and their interactions that can be used to build-up more complex hypotheses. Currently, we rely on benchmark performance or vague intuitive descriptions to pin-point specific phenomena, which means most hypotheses rely on imprecise vocabulary that won’t stand the test of time.

In the short-term, I’m interested in thinking how we can map out what models can and can’t do, which I believe will naturally relate to long-form generation. It is incredibly difficult to evaluate long-form generation rigorously, and it is hard to show long-form generations in power point slides, which has made coordinating the issues in long-form generation difficult for the academic community. In the medium-term, I think we need to tackle the non-objective aspects of language, as almost all communication is open to interpretation, relying instead on the pragmatic attempt at cooperation to bridge this gap. Focusing on easy-to-evaluate aspects of language doesn’t do it justice. Perhaps looking at indirect evaluation, where we evaluate what generated language can be used for, rather than whether it is “correct” can help move researchers in that direction. My long-term goal is to create discursive machines.