New Features:

• High-performance sparse quantized convolutional neural networks supported on AVX2 systems.
• CCX detection added to the DeepSparse Engine for AMD systems.
• deepsparse.server integration and CLIs added with Hugging Face transformers pipelines support.

Changes:

Performance improvements made for

• FP32 sparse BERT models
• batch size 1 networks
• quantized sparse BERT models
• Pooling operations

Resolved Issues:

• When hyperthreads are disabled in the BIOS, core/socket information on certain systems can now be detected.
• Hugging Face transformers validation flows for QQP now giving correct accuracy metrics.
• PyTorch downloaded for YOLO model stubs now supported.

Known Issues:

• When running NanoDet-Plus-m, the DeepSparse Engine will fail with an assertion (See #279). A hotfix is being pursued.