Portrait of Mohammad Aflah Khan

Mohammad Aflah Khan

PhD Student @ MPI-SWS · Research Scientist @ EleutherAI

Hi, I’m Aflah, a first year PhD student at the Max Planck Institute for Software Systems. My work centers on deepening our understanding of large language models (LLMs) and rigorously evaluating their capabilities. I’m also passionate about the systems side of LLMs, with hands-on experience in large-scale pretraining and inference. In the past, I’ve contributed to projects targeting hate speech reduction and other NLP applications for social good.

Open to internships in LLM research for summer 2027
Contact me

Latest News

Apr 2026 ✈️ Catch me in Seoul, South Korea for ICML 2026.

Jun 2026 📝 Position: Don't Just "Fix it in Post": A Science of AI Must Study Learning Dynamics accepted as an oral at ICML 2026 and accepted at the Mechanistic Interpretability Workshop @ ICML 2026.

Apr 2026 📝 Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective accepted as an oral at ACL 2026. 🏆 We also won the Senior Area Chair Highlight Award.

Apr 2026 ✈️ Catch me in Rio de Janeiro, Brazil for ICLR 2026.

Mar 2026 📄 Fractional Rotation, Full Potential? Investigating Performance and Convergence of Partial RoPE released as a preprint.

Experience

Ongoing first (rest sorted by end date)

Max Planck Institute for Software Systems (MPI-SWS)

PhD Student [Full Time] • Sept, 2026 — Present

Research Software Engineer [Full Time] • April, 2024 — July, 2026

Research Software Engineer [Part Time] • Nov, 2023 — March, 2024

Research Intern [Part Time] • Aug, 2023 — Oct, 2023

I am a first-year PhD student at MPI-SWS, advised by Dr. Krishna P. Gummadi and Dr. Abhilasha Ravichander. I am a member of the Networked Systems Research Group and QUEST Lab. My research examines the capabilities and limitations of large language models (LLMs), with an emphasis on improving their performance and reliability. Previously, as a Research Software Engineer, I worked on pre-training and inference optimization, tool use and reasoning, memorization, parameter-efficient fine-tuning (PEFT), and factual knowledge acquisition and evaluation. I also built and maintained OpenChat, an internal chatbot; MaxCast, a research paper-to-podcast service; and MaxChat, a document-based conversational system, including their on-premises model deployment and performance optimization.

EleutherAI

Research Scientist [Part Time] • June, 2026 — Present

Community Researcher [Part Time] • Dec, 2022 — June, 2026

I'm currently working on improving GPT-NeoX, EleutherAI's pre-training codebase. I'm also collaborating with the ATLAS project at CERN to build better jet tagging classifiers for di-higgs boson events. I've previously worked on -

  • Pythia - A Suite for Analyzing Large Language Models Across Training and Scaling (Accepted ICML'23 as an Oral) - Majorly contributed to the gender bias evals and intervention case study. The models have over 18 million downloads (as of April 2025)
  • Recite, Reconstruct, Recollect - Memorization in LMs as a Multifaceted Phenomenon (Accepted ICLR'25) - An intuitive taxonomy to classify memorized sequences and then build predictors based on these classes
  • Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics (Accepted ICML'26 as an Oral) - A position paper arguing for the importance of studying training dynamics to arrive at a science of AI
I was also featured in the first EleutherAI community spotlight.

Laboratory for Computational Social Systems (LCS2)

Undergraduate Student Researcher • June, 2021 — May, 2024

At LCS2, I worked on projects spanning hate-speech normalization and fine-tuning strategies for more effective hate-speech detection. I also co-led QUENCH, a benchmark designed to evaluate advanced reasoning capabilities in large language models, with a particular focus on Indic contexts.

Publications

* indicates equal contribution

Poster

Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander

ICLR 2026 - The Fourteenth International Conference on Learning Representations
IASEAI 2026 - The International Association for Safe & Ethical AI (Non-Archival)
(An earlier version of this work was presented at R2-FM, ICML 2025)

Press/Other Coverage Science MIT Technology Review China (DeepTech)

Johnny Tian-Zheng Wei*, Ameya Godbole*, Mohammad Aflah Khan*, Ryan Wang, Xiaoyuan Zhu, James Flemings, Nitya Kashyap, Krishna P. Gummadi, Willie Neiswanger, Robin Jia

[Oral] ICLR 2026 - The Fourteenth International Conference on Learning Representations (Top ~1.2% of submissions)
Data-FM @ ICLR 2026 - Workshop on Navigating and Addressing Data Problems for Foundation Models (Non-Archival)

Qinyuan Wu, Soumi Das, Mahsa Amani, Bishwamittra Ghosh, Mohammad Aflah Khan, Krishna P. Gummadi, Muhammad Bilal Zafar

ICLR 2026 - The Fourteenth International Conference on Learning Representations
IASEAI 2026 - The International Association for Safe & Ethical AI (Non-Archival)
(An earlier version of this work was presented at MemFM, ICML 2025)

Poster

Soumi Das, Camila Kolling, Mohammad Aflah Khan, Mahsa Amani, Bishwamittra Ghosh, Qinyuan Wu, Till Speicher, Krishna P. Gummadi

IASEAI 2026 - The International Association for Safe & Ethical AI (Non-Archival)

Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah, Catherine Arnett, Fazl Barez, Naomi Saphra

[Oral] ICML 2026 (Position Paper Track) - Forty-Third International Conference on Machine Learning (Top ~1.6% of submissions)
Mechanistic Interpretability Workshop @ ICML 2026 (Non-Archival)

🏆 Senior Area Chair Highlight Award

Bishwamittra Ghosh, Soumi Das, Till Speicher, Qinyuan Wu, Mohammad Aflah Khan, Deepak Garg, Krishna P. Gummadi, Evimaria Terzi

[Oral] ACL 2026 - The 64th Annual Meeting of the Association for Computational Linguistics (Top ~4.1% of submissions)

Mohammad Aflah Khan, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander

Under Review

Poster

Mohammad Aflah Khan*, Ameya Godbole*, Johnny Tian-Zheng Wei, Ryan Wang, James Flemings, Krishna P. Gummadi, Willie Neiswanger, Robin Jia

EMNLP 2025 System Demonstrations - The 2025 Conference on Empirical Methods in Natural Language Processing

Poster

USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, Naomi Saphra

ICLR 2025 - The Thirteenth International Conference on Learning Representations

Qinyuan Wu, Mohammad Aflah Khan, Soumi Das, Vedant Nanda, Bishwamittra Ghosh, Camila Kolling, Till Speicher, Laurent Bindschaedler, Krishna P Gummadi, Evimaria Terzi

WSDM 2025 - Proceedings of the 18th ACM International Conference on Web Search and Data Mining

Mohammad Aflah Khan*, Neemesh Yadav*, Sarah Masud, Md Shad Akhtar

COLING 2025 - Proceedings of the 31st International Conference on Computational Linguistics

Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander

R2-FM @ ICML 2025 - Workshop on Reliable and Responsible Foundation Models (Non-Archival)

Bishwamittra Ghosh, Soumi Das, Qinyuan Wu, Mohammad Aflah Khan, Krishna P. Gummadi, Evimaria Terzi, Deepak Garg

MemFM @ ICML 2025 - The Impact of Memorization on Trustworthy Foundation Models (Non-Archival)

Qinyuan Wu, Soumi Das, Mahsa Amani, Bishwamittra Ghosh, Mohammad Aflah Khan, Krishna P. Gummadi, Muhammad Bilal Zafar

MemFM @ ICML 2025 - The Impact of Memorization on Trustworthy Foundation Models (Non-Archival)

Mohammad Aflah Khan*, Neemesh Yadav*, Diksha Sethi*, Raghav Sahni*

The Second Tiny Papers Track at ICLR 2024

Sarah Masud*, Mohammad Aflah Khan*, Vikram Goyal, Md Shad Akhtar, Tanmoy Chakraborty

EACL 2024 - Findings of the Association for Computational Linguistics

Till Speicher, Mohammad Aflah Khan, Qinyuan Wu, Vedant Nanda, Soumi Das, Bishwamittra Ghosh, Krishna P. Gummadi, Evimaria Terzi

Shrey Satapara, Sarah Masud, Hiren Madhu, Mohammad Aflah Khan, Md Shad Akhtar, Tanmoy Chakraborty, Sandip Modha, Thomas Mandl

FIRE 2023 - Proceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation

Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar, Tanmoy Chakraborty

In Working Notes of FIRE 2023 - Forum for Information Retrieval Evaluation

Press/Other Coverage KDNuggets HackerNews AWS Blog Data Science Gems (YT)

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal

[Oral] ICML 2023 - The Fortieth International Conference on Machine Learning (Top ~2.3% of submissions)

Mohammad Aflah Khan*, Neemesh Yadav*, Mohit Jain, Sanyam Goyal

The First Tiny Papers Track at ICLR 2023

Neemesh Yadav*, Mohammad Aflah Khan*, Diksha Sethi, Raghav Sahni

The First Tiny Papers Track at ICLR 2023

Poster

Sarah Masud, Manjot Bedi, Mohammad Aflah Khan, Md Shad Akhtar, Tanmoy Chakraborty

KDD 2022 - Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Applied Data Science Track)

Talks

An Overview of AlphaFold

Max Planck Institute for Software Systems: MPI-SWS (Internal Paper Reading Group) • February, 2026

[Paper Reading & Discussion] Metadata Conditioning Accelerates Language Model Pre-training (MeCo)

Max Planck Institute for Software Systems: MPI-SWS (Internal Paper Reading Group) • September, 2025

[Talk] LLMs at Scale

Max Planck Computing and Data Facility: MPCDF (AI Kick-off Workshop) • April, 2025

Max Planck Institute for the Science of Light (Hosted by Florian Marquardt) • April, 2025

[Talk] An Overview of DeepSeek-{V3/R1}

Max Planck Institute for Software Systems: MPI-SWS (Part of AI, Computing & Society Initiative) • February, 2025

[Talk] Democratizing and Accelerating Research with LLMs: Making Science More Accessible Whilst Finding Interesting Research Problems

Max Planck Institute for Security and Privacy: MPI-SP (Hosted by Meeyoung Cha) • December, 2024

Social Links

  • Github: https://github.com/aflah02
  • Twitter: http://twitter.com/aflah02101
  • LinkedIn: https://www.linkedin.com/in/mohammad-aflah-khan
  • Website: https://aflah02.github.io/