Nayoung Lee | 이나영

Welcome!

Hi! I am a Ph.D. candidate in the department of Electrical and Computer Engineering (ECE) at University of Wisconsin-Madison, advised by Prof. Dimitris Papailiopoulos and Prof. Kangwook Lee. I received my M.S. in 2018 from Seoul National University, where I was advised by Prof. Jungwoo Lee and learned about communication systems. I also received my B.S. in ECE from Seoul National University. I am a recipient of the Korean Government Scholarship Program for Study Overseas.

I am open to research collaboration and internship opportunities.

Research Interest

I am interested in machine learning, with the focus on finding efficient, robust and scalable machine learning algorithm. Recently, I’ve been interested in understanding the algorithmic capabilities of large language models and ways to improve performance through data, prompting, inference-time methods, and verifiers. Previously, I have worked on neural network pruning.

News

Publications

Ongoing Projects

Teaching Arithmetic to Small Transformers

LLMs trained on vast amounts of data, eventually learn basic arithmetic, even when these tasks are not explicitly encoded in the next-prediction loss objective. To untangle various factors in play, we investigate the arithmetic capabilities of small Transformer models. We find the importance of data sampling, formatting, and prompting for the model to elicit arithmetic capabilities.

Super-Seeds: Exploring the trade-off in storage and computation

We explore the trade-off between extreme model compression and computation of neural networks. We aim to answer the following question: For a given level of model compression, what is the minimum computation required to recover a high accuracy model? We discover that one can trade off some accuracy of the model with significant gains in storage cost, at a relatively small decompression cost.

Teaching Experience

Graduate Teaching Assistant

Undergraduate Teaching Assistant

Talks