Igor Fedorov

MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment

Hanxian Huang*, Igor Fedorov*, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghuraman Krishnamoorthi

Short Data, Long Context: Distilling Positional Knowledge in Transformers

Patrick Huber, Ernie Chang, Chinnadhurai Sankar, Rylan Conway, Igor Fedorov, Md Rifat Arefin, Adithya Sagar

MobileLLM-Pro Technical Report

Patrick Huber, Ernie Chang, Wei Wen, Igor Fedorov, Tarek Elgamal, Hanxian Huang, Naveen Suda, Chinnadhurai Sankar, Vish Vogeti, Yanghan Wang, Alex Gladkov, Kai Sheng Tai, Abdelrahman Elogeel, Tarek Hefny, Vikas Chandra, Ahmed Aly, Anuj Kumar, Raghuraman Krishnamoorthi, Adithya Sagar

SpinQuant: LLM Quantization with Learned Rotations

Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, Tijmen Blankevoort

Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations

Igor Fedorov, Kate Plawiak, Lemeng Wu, Tarek Elgamal, Naveen Suda, Eric Smith, Hongyuan Zhan, Jianfeng Chi, Yuriy Hulovatyy, Kimish Patel, Zechun Liu, Changsheng Zhao, Yangyang Shi, Tijmen Blankevoort, Mahesh Pasupuleti, Bilge Soran, Zacharie Delpierre Coudert, Rachad Alao, Raghuraman Krishnamoorthi, Vikas Chandra

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Z. Liu, C. Zhao, F. Iandola, C. Lai, Y. Tian, I. Fedorov, Y. Xiong, E. Chang, Y. Shi, R. Krishnamoorthi, L. Lai, V. Chandra

Dictionaries in Machine Learning

K. Kreutz-Delgado, B. D. Rao, I. Fedorov, S. Das

Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale

W. Wen, K. Liu, I. Fedorov, X. Zhang, H. Yin, W. Chu, K. Hassani, M. Sun, J. Liu, X. Wang, L. Jiang, Y. Chen, B. Zhang, X. Liu, D. Cheng, Z. Chen, G. Zhao, F. Han, J. Yang, Y. Hao, L. Xiong, W. Chen

SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscape

H. Zheng, K. Liu, I. Fedorov, X. Zhang, W. Chen, W. Wen

DistDNAS: Search Efficient Feature Interactions within 2 Hours

T. Zhang, W. Wen, I. Fedorov, X. Liu, B. Zhang, F. Han, W. Chen, Y. Han, F. Yan, H. Li, Y. Chen

PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices

Y. Chai, D. Tripathy, C. Zhou, D. Gope, I. Fedorov, R. Matas, D. Brooks, G. Wei, P. Whatmough

Efficient Edge Inference by Selective Query

A. Kag, I. Fedorov, A. Gangrade, P. N. Whatmough, V. Saligrama

Error Detection

M. Haddon, I. Fedorov, R. Jeyapaul, P. N. Whatmough, Z. Liu

UDC: Unified DNAS for Compressible TinyML Models

I. Fedorov, R. M. Navarro, H. Tann, C. Zhou, M. Mattina, P. N. Whatmough

Restructurable Activation Networks

K. Bhardwaj, J. Ward, C. Tung, D. Gope, L. Meng, I. Fedorov, A. Chalfin, P. Whatmough, D. Loh

Achieving High TinyML Accuracy through Selective Cloud Interactions

A. Kag, I. Fedorov, A. Gangrade, P. N. Whatmough, V. Saligrama

Neural Network System and Training Method

I. Fedorov, P. Whatmough

A Unified Neural Network Optimization Framework

I. Fedorov, R. Matas, C. Zhou, H. Tann, P. Whatmough, M. Mattina

System, Devices and/or Processes for Defining a Search Space for Neural Network Processing Device Architectures

H. Tann, R. Navarro, I. Fedorov, C. Zhou, P. Whatmough, M. Mattina

MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers

C. Banbury*, C. Zhou*, I. Fedorov*, R. M. Navarro, U. Thakker, D. Gope, V. J. Reddi, M. Mattina, P. N. Whatmough

Image Denoising Neural Network Architecture and Method of Training the Same

M. El-Khamy, I. Fedorov, J. Lee

TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids

I. Fedorov, M. Stamenovic, C. Jensen, L. Yang, A. Mandell, Y. Gan, M. Mattina, P. N. Whatmough

SSGD: Sparsity-promoting Stochastic Gradient Descent Algorithm for Unbiased DNN Pruning

C. Lee, I. Fedorov, B. D. Rao, H. Garudadri

Mango: A Python Library for Parallel Hyperparameter Tuning

S. Sandha, M. Aggarwal, I. Fedorov, M. Srivastava

SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

I. Fedorov, R. P. Adams, M. Mattina, P. N. Whatmough

Pushing the Limits of RNN Compression

U. Thakker, I. Fedorov, J. Beu, D. Gope, C. Zhou, G. Dasika, M. Mattina

Compressing RNNs for IoT Devices by 15-38x using Kronecker Products

U. Thakker, J. Beu, D. Gope, C. Zhou, I. Fedorov, G. Dasika, M. Mattina

Structured Learning with Scale Mixture Priors

I. Fedorov

Multimodal Sparse Bayesian Dictionary Learning

I. Fedorov, B. D. Rao

Multimodal Sparse Bayesian Dictionary Learning Applied to Multimodal Data Classification

I. Fedorov, B. D. Rao, T. Q. Nguyen

Re-weighted Learning for Sparsifying Deep Neural Networks

I. Fedorov, B. D. Rao

A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem

I. Fedorov, A. Nalci, R. Giri, B. D. Rao, T. Q. Nguyen, H. Garudadri

Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem

A. Nalci, I. Fedorov, M. Al-Shoukairi, T. T. Liu, B. D. Rao

Relevance Subject Machine: A Novel Person Re-Identification Framework

I. Fedorov, R. Giri, B. D. Rao, T. Q. Nguyen

Robust Bayesian Method for Simultaneous Block Sparse Signal Recovery with Applications to Face Recognition

I. Fedorov, R. Giri, B. D. Rao, T. Q. Nguyen

Image Reconstruction under Imaging Time Constraints

I. Fedorov, S. Obrzut, B. Song, B. D. Rao

Total Variation Regularization in I-123 Ioflupane SPECT Reconstruction

I. Fedorov, B. Song, B. D. Rao, I. Levitan, S. Obrzut

Kinect Depth Video Compression for Action Recognition

I. Fedorov

Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from Long Sequences of RGB-D Images

A. Khosrowpour, I. Fedorov, A. Holynski, J. C. Niebles, M. Golparvar-Fard

Power Delivery for Series Connected Voltage Domains in Digital Circuits

P. Shenoy, I. Fedorov, T. Neyens, P. Krein