v1.0a
版本发布时间: 2020-08-11 20:14:20
intel/neural-compressor最新发布版本:v2.6(2024-06-14 21:55:11)
Intel® Low Precision Optimization Tool (iLiT) is an open-sourced python library which is intended to deliver a unified low-precision inference solution cross multiple Intel optimized DL frameworks on both CPU and GPU. It supports automatic accuracy-driven tuning strategies, along with additional objectives like performance, model size, or memory footprint. It also provides the easy extension capability for new backends, tuning strategies, metrics and objectives.
Feature List:
- Unified low precision quantization interface cross multiple Intel optimized frameworks (TensorFlow, PyTorch, and MXNet)
- Built-in tuning strategies, including Basic, Bayesian, and MSE
- Built-in evaluation metrics, including TopK (image classification), F1 (NLP), and CocoMAP (object detection)
- Built-in tuning objectives, including Performance, ModelSize, and Footprint
- Extensible API design to add new strategy, framework backend, metric, and objective
- KL-divergence calibration for TensorFlow and MXNet
- Tuning process resume from certain checkpoint
Supported Models:
| Model | Framework | Model | Framework | Model | Framework |
|---|---|---|---|---|---|
| ResNet50 V1 | MXNet | BERT-Large RTE | PyTorch | ResNet18 | PyTorch |
| MobileNet V1 | MXNet | BERT-Large QNLI | PyTorch | ResNet50 V1 | TensorFlow |
| MobileNet V2 | MXNet | BERT-Large CoLA | PyTorch | ResNet50 V1.5 | TensorFlow |
| SSD-ResNet50 | MXNet | BERT-Base SST-2 | PyTorch | ResNet101 | TensorFlow |
| SqueezeNet V1 | MXNet | BERT-Base RTE | PyTorch | Inception V1 | TensorFlow |
| ResNet18 | MXNet | BERT-Base STS-B | PyTorch | Inception V2 | TensorFlow |
| Inception V3 | MXNet | BERT-Base CoLA | PyTorch | Inception V3 | TensorFlow |
| DLRM | PyTorch | BERT-Base MRPC | PyTorch | Inception V4 | TensorFlow |
| BERT-Large MRPC | PyTorch | ResNet101 | PyTorch | Inception ResNet V2 | TensorFlow |
| BERT-Large SQUAD | PyTorch | ResNet50 V1.5 | PyTorch | SSD ResNet50 V1 | TensorFlow |
Known Issues:
- Statistics collection for KL algorithm is slow in TensorFlow due to lack of tensor inspector APIs
- MSE tuning strategy is not supported in PyTorch
Validated Configurations:
- Python 3.6 & 3.7
- Centos 7
- TensorFlow 1.15, 2.0 and 2.1
- PyTorch 1.5
- MxNet 1.6
Distribution:
| Channel | Links | Install Command | |
|---|---|---|---|
| Source | Github | https://github.com/intel/lp-opt-tool.git | $ git clone https://github.com/intel/lp-opt-tool.git |
| Binary | Pip | https://pypi.org/project/ilit | $ pip install ilit |
| Binary | Conda | https://anaconda.org/intel/ilit | $ conda config --add channels intel $ conda install ilit |
Contact:
Please feel free to contact ilit.maintainers@intel.com, if you get any questions.