Igor Fedorov

I am an AI Research Scientist at Meta, where I am responsible for model optimization research and infrastructure development for family of apps and mixed reality ML models. I have also been responsible for algorithm research and software development for neural architecture search on ads recommendation models. Before that, I was a Staff Research Engineer in the machine learning research group at ARM Research. I completed my PhD in Electrical Engineering in 2018 at the University of California San-Diego. I was supervised by Prof. Bhaskar Rao and co-advised by Prof. Truong Nguyen. Before UCSD, I completed my B.S. and M.S. (supervised by Prof. Pierre Moulin) in Electrical Engineering at the University of Illinois Urbana-Champaign.

I am lucky to be married to Katarina Fedorov.

cv / email / google scholar / linkedin

Research / Patents

	SpinQuant: LLM Quantization with Learned Rotations Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, Tijmen Blankevoort Arxiv, 2024 [Paper]
	MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Z. Liu, C. Zhao, F. Iandola, C. Lai, Y. Tian, I. Fedorov, Y. Xiong, E. Chang, Y. Shi, R. Krishnamoorthi, L. Lai, V. Chandra ICML, 2024 [Paper]
	Dictionaries in machine learning K. Kreutz-Delgado, B.D. Rao, I. Fedorov, S. Das Signal Processing and Machine Learning Theory, 2024 [Book]
	Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale W. Wen, K. Liu, I. Fedorov, X. Zhang, H. Yin, W. Chu, K. Hassani, M. Sun, J. Liu, X. Wang, L. Jiang, Y. Chen, B. Zhang, X. Liu, D. Cheng, Z. Chen, G. Zhao, F. Han, J. Yang, Y. Hao, L. Xiong, W. Chen Arxiv, 2023 [Paper]
	SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscape H. Zheng, K. Liu, I. Fedorov, X. Zhang, W. Chen, W. Wen Arxiv, 2023 [Paper]
	DistDNAS: Search Efficient Feature Interactions within 2 Hours T. Zhang, W. Wen, I. Fedorov, X. Liu, B. Zhang, F. Han, W. Chen, Y. Han, F. Yan, H. Li, Y. Chen Arxiv, 2023 [Paper]
	PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices Y. Chai, D. Tripathy, C. Zhou, D. Gope, I. Fedorov, R. Matas, D. Brooks, G. Wei, P. Whatmough ArXiv, 2023 [paper]
	Efficient Edge Inference by Selective Query A. Kag, I. Fedorov, A. Gangrade, P. N. Whatmough, V. Saligrama ICLR, 2023 [paper]
	Error detection M. Haddon, I. Fedorov, R. Jeyapaul, P. N. Whatmough, Z. Liu US Patent Application, 2022
	UDC: Unified DNAS for Compressible TinyML Models I. Fedorov, R. M. Navarro, H. Tann, C. Zhou, M. Mattina, P. N. Whatmough NeurIPS, 2022 [paper]
	Restructurable Activation Networks K. Bhardwaj, J. Ward, C. Tung, D. Gope, L. Meng, I. Fedorov, A. Chalfin, P. Whatmough, D. Loh ArXiv, 2022 [paper]
	Achieving High TinyML Accuracy through Selective Cloud Interactions A. Kag, I. Fedorov, A. Gangrade, P. N. Whatmough, V. Saligrama ICML DyNN workshop, 2022 [paper]
	Neural network system and training method I. Fedorov, P. Whatmough US Patent Application, 2022
	A unified neural network optimization framework I. Fedorov, R. Matas, C. Zhou, H. Tann, P. Whatmough, M. Mattina US Patent Application, 2022
	MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers C. Banbury, C. Zhou, I. Fedorov, R. M. Navarro, U. Thakker, D. Gope, V. J. Reddi, M. Mattina, P. N. Whatmough MLSys, 2021* [paper] [models] [blog1] [blog2]
	Image denoising neural network architecture and method of training the same M. El-Khamy, I. Fedorov, J. Lee US Patent, 2020 [patent]
	TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids I. Fedorov, M. Stamenovic, C. Jensen, L. Yang, A. Mandell, Y. Gan, M. Mattina, P. N. Whatmough INTERSPEECH, 2020 [paper]
	SSGD: Sparsity-promoting Stochastic Gradient Descent Algorithm for Unbiased DNN Pruning C. Lee, I. Fedorov, B. D. Rao, H. Garudadri IEEE International Conference on Acoustics, Speech, and Signal Processing, 2020 [paper]
	Mango: A Python Library for Parallel Hyperparameter Tuning S. Sandha, M. Aggarwal, I. Fedorov, M. Srivastava IEEE International Conference on Acoustics, Speech, and Signal Processing, 2020 [paper]
	SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers I. Fedorov, R. P. Adams, M. Mattina, P. N. Whatmough Proc. of the Conference on Neural Information Processing Systems (NeurIPS), 2019 [paper] [youtube]
	Pushing the limits of RNN Compression U. Thakker, I. Fedorov, J. Beu, D. Gope, C. Zhou, G. Dasika, M. Mattina Workshop on Energy Efficient Machine Learning and Cognitive Computing, 2019 [paper]
	Compressing RNNs for IoT devices by 15-38x using Kronecker Products U. Thakker, J. Beu, D. Gope, C. Zhou, I. Fedorov, G. Dasika, M. Mattina arXiv, 2019 [paper]
	Structured Learning with Scale Mixture Priors I. Fedorov PhD Thesis [paper]
	Multimodal Sparse Bayesian Dictionary Learning I. Fedorov, B. D. Rao Arxiv preprint [paper]
	Multimodal Sparse Bayesian Dictionary Learning Applied to Multimodal Data Classification I. Fedorov, B. D. Rao, T. Q. Nguyen IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016 [paper]
	Re-weighted Learning for Sparsifying Deep Neural Networks I. Fedorov, B. D. Rao Arxiv preprint, 2018 [paper]
	A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem I. Fedorov, A. Nalci, R. Giri, B. D. Rao, T. Q. Nguyen, H. Garudadri Signal Processing, 2018 [paper] [code]
	Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem A. Nalci, I. Fedorov, M. Al-Shoukairi, T. T. Liu, B. D. Rao IEEE Transactions on Signal Processing, 2018 [paper]
	Relevance Subject Machine: A Novel Person Re-Identification Framework I. Fedorov, R. Giri, B. D. Rao, T. Q. Nguyen arxiv preprint, 2017 [paper]
	Robust Bayesian Method for Simultaneous Block Sparse Signal Recovery with Applications to Face Recognition I. Fedorov, R. Giri, B. D. Rao, T. Q. Nguyen IEEE International Conference on Image Processing (ICIP), 2016 [paper]
	Image Reconstruction under Imaging Time Constraints I. Fedorov, S. Obrzut, B. Song, B. D. Rao 51st Asilomar Conference on Signals, Systems, and Computers, 2017 [paper]
	Total Variation Regularization in I-123 Ioflupane SPECT Reconstruction I. Fedorov, B. Song, B. D. Rao, I. Levitan, S. Obrzut Journal of Nuclear Medicine, 2017 [paper]
	Kinect depth video compression for action recognition I. Fedorov M.S. Thesis, 2014 [paper]
	Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from long sequences of RGB-D Images A. Khosrowpour, I. Fedorov, A. Holynski, J. C. Niebles, M. Golparvar-Fard Construction Research Congress, 2014 [paper]
	Power delivery for series connected voltage domains in digital circuits P. Shenoy, I. Fedorov, T. Neyens, P. Krein International Conference on Energy Aware Computing, 2011 [paper]