Max Planck Institute for Software Systems (MPI-SWS)
Research Software Engineer • April, 2024 — Present [Full Time] | Nov, 2023 — March, 2024 [Part time] | Aug, 2023 — Oct, 2023 [Intern]
Working under Dr Krishna Gummadi to explore different aspects of LLMs. Some areas we've explored/are exploring are
- LLM memorization and the impact of Parameter-Efficient Fine-Tuning (PEFT) on memorization
- Knowledge acquisition and evaluation of factual knowledge in LLMs
- Retrieval-Augmented Generation (RAG) architectures
- Optimizing pre-training and inference for LLMs
- Built and currently maintain key internal tools OpenChat (An internal chatbot), MaxCast (A research paper-to-podcast conversion service) & MaxChat (A document-based chat service). These services were developed from scratch, including hosting models on-premises and fine-tuning for optimal performance.
- Published and submitted research to top-tier (A*) conferences
EleutherAI
Open Source Contributor • Dec, 2022 — Present
Working on predicting memorization behavior in LLMs by finding which strings from the training data will be memorized. Previously worked on the Pythia model suite.
Laboratory for Computational Social Systems (LCS2)
Undergraduate Student Researcher • June, 2021 — May, 2024
I've worked on a variety of projects, from hate speech normalization to designing recommendations for fine-tuning improved hate speech detectors. I also led the QUENCH project, a benchmark aimed at evaluating advanced reasoning abilities in large language models, with a particular emphasis on Indic contexts.
Goldman Sachs
Summer Analyst • May, 2023 — July, 2023
Worked in the Finance, Planning & Analysis Engineering division towards revamping the central hub of the department. Also built POCs based on user feedback to improve the search and access experience on the webapp. Also recieved a return offer to join full time as an Analyst.
Google Summer of Code - TensorFlow
Open Source Developer • May, 2022 — Sept, 2022
Worked with Matthew Watson & Chen Qian towards adding support for data augmentation layers to KerasNLP a library under the Keras/TensorFlow Ecosystem which aims to build industry oriented NLP Solutions. I also contributed to several bug fixes and other utilities such as tokenizers and transformer encoder & decoder.
Indraprastha Institute of Information Technology (IIIT-D)
B.Tech. in Computer Science and Engineering • 2020 — 2024
- Dean's List for Academic Excellence (2022-23)
- Dean's List for Innovation in Research and Development (2022-23)
- Dean's List for Academic Excellence (2021-22)
GPA - 9.63/10 [Dept. Rank 2 & Batch Rank 3]
Lal Bahadur Shastri School
Senior-Secondary Education (12th Grade) • 2020
Secured 95% in All India Senior School Certificate Examination
Banyan Tree School
Secondary Education (10th Grade) • 2018
Secured 95.8% in All India Secondary School Examination
USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, Naomi Saphra • 2025
ICLR 2025 - The Thirteenth International Conference on Learning Representations
Qinyuan Wu, Mohammad Aflah Khan, Soumi Das, Vedant Nanda, Bishwamittra Ghosh, Camila Kolling, Till Speicher, Laurent Bindschaedler, Krishna P Gummadi, Evimaria Terzi • 2025
WSDM 2025 - Proceedings of the 18th ACM International Conference on Web Search and Data Mining
Mohammad Aflah Khan*, Neemesh Yadav*, Sarah Masud, Md Shad Akhtar • 2025
COLING 2025 - Proceedings of the 31st International Conference on Computational Linguistics
Soumi Das, Camila Kolling, Mohammad Aflah Khan, Mahsa Amani, Bishwamittra Ghosh, Qinyuan Wu, Till Speicher, Krishna P. Gummadi • 2025
Under Review
Till Speicher, Mohammad Aflah Khan, Qinyuan Wu, Vedant Nanda, Soumi Das, Bishwamittra Ghosh, Krishna P. Gummadi, Evimaria Terzi • 2024
Under Review
Mohammad Aflah Khan*, Neemesh Yadav*, Diksha Sethi*, Raghav Sahni* • 2024
The Second Tiny Papers Track at ICLR 2024
Sarah Masud*, Mohammad Aflah Khan*, Vikram Goyal, Md Shad Akhtar, Tanmoy Chakraborty • 2024
EACL 2024 - Findings of the Association for Computational Linguistics
Shrey Satapara, Sarah Masud, Hiren Madhu, Mohammad Aflah Khan, Md Shad Akhtar, Tanmoy Chakraborty, Sandip Modha, Thomas Mandl • 2023
FIRE 2023 - Proceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation
Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar, Tanmoy Chakraborty • 2023
In Working Notes of FIRE 2023 - Forum for Information Retrieval Evaluation
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal • 2023
ICML 2023 - The Fortieth International Conference on Machine Learning
Mohammad Aflah Khan*, Neemesh Yadav*, Mohit Jain, Sanyam Goyal • 2023
The First Tiny Papers Track at ICLR 2023
Neemesh Yadav*, Mohammad Aflah Khan*, Diksha Sethi, Raghav Sahni • 2023
The First Tiny Papers Track at ICLR 2023
Sarah Masud, Manjot Bedi, Mohammad Aflah Khan, Md Shad Akhtar, Tanmoy Chakraborty • 2022
KDD 2022 - Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
[Talk] LLMs at Scale
Max Planck Computing and Data Facility: MPCDF (AI Kick-off Workshop) • April, 2025
Max Planck Institute for the Science of Light (Hosted by Florian Marquardt) • April, 2025
Max Planck Institute for Software Systems: MPI-SWS (Part of AI, Computing & Society Initiative) • February, 2025
[Talk] Democratizing and Accelerating Research with LLMs: Making Science More Accessible Whilst Finding Interesting Research Problems
Max Planck Institute for Security and Privacy: MPI-SP (Hosted by Meeyoung Cha) • December, 2024
[Demo + Lightning Talk] Empowering Research with Open-Access LLMs: From Tools to Copilots
AI, Computing & Society Initiative Launch Event (At Max Planck Institute for Software Systems: MPI-SWS) • December, 2024
Max Planck Institute for Software Systems: MPI-SWS (Internal Paper Reading Group) • July, 2024
Max Planck Institute for Software Systems: MPI-SWS (Internal Paper Reading Group) • May, 2024
[Talk] Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Max Planck Institute for Software Systems: MPI-SWS (Hosted by Krishna Gummadi) • July, 2023
Goldman Sachs (Internal NLP/IR Reading Group) • June, 2023