Summary of major features and improvements  

• More Generative AI coverage and framework integrations to minimize code changes.

  • Expanded model support for direct PyTorch model conversion – automatically convert additional models directly from PyTorch or execute via torch.compile with OpenVINO as the backend.
  • New and noteworthy models supported – we have enabled models used for chatbots, instruction following, code generation, and many more, including prominent models like LLaVA, chatGLM, Bark (text to audio), and LCM (Latent Consistency Models, an optimized version of Stable Diffusion).
  • Easier optimization and conversion of Hugging Face models – compress LLM models to Int8 with the Hugging Face Optimum command line interface and export models to the OpenVINO IR format.
  • OpenVINO is now available on Conan – a package manager which enables more seamless package management for large-scale projects for C and  C++ developers.

• Broader Large Language Model (LLM) support and more model compression techniques.

  • Accelerate inference for LLM models on Intel® Core™ CPU and iGPU with the use of Int8 model weight compression.
  • Expanded model support for dynamic shapes for improved performance on GPU.
  • Preview support for Int4 model format is now included. Int4 optimized model weights are now available to try on Intel® Core™ CPU and iGPU, to accelerate models like Llama 2 and chatGLM2.
  • The following Int4 model compression formats are supported for inference in runtime:
    • Generative Pre-training Transformer Quantization (GPTQ); with GPTQ-compressed models, you can access them through the Hugging Face repositories.
    • Native Int4 compression through Neural Network Compression Framework (NNCF).

• More portability and performance to run AI at the edge, in the cloud, or locally.

  • In 2023.1 we announced full support for ARM architecture, now we have improved performance by enabling FP16 model formats for LLMs and integrating additional acceleration libraries to improve latency.

Support Change and Deprecation Notices

• The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated and will be removed from installation options and distribution channels with 2025.0. To learn more, refer to the OpenVINO Legacy Features and Components page. To ensure optimal performance, install the OpenVINO package (pip install openvino), which includes essential components such as OpenVINO Runtime, OpenVINO Converter, and Benchmark Tool.
• Tools: 
  • Deployment Manager is deprecated and will be removed in the 2024.0 release.
  • Accuracy Checker is deprecated and will be discontinued with 2024.0.   
  • Post-Training Optimization Tool (POT)  is deprecated and will be discontinued with 2024.0. 
  • Model Optimizer is deprecated and will be fully supported up until the 2025.0 release. Model conversion to the OpenVINO IR format should be performed through OpenVINO Model Converter which is part of the PyPI package. Follow the Model Optimizer to OpenVINO Model Converter transition guide for smoother transition. Known limitations are TensorFlow model with TF1 Control flow and object detection models. These limitations relate to the gap in TensorFlow direct conversion capabilities which will be addressed in upcoming releases.
  • PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF).
• Runtime: 
  • Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future release. We encourage developers to use the Neural Processing Unit (NPU) for low powered systems like Intel® Core™ Ultra or 14th generation and beyond.  
  • OpenVINO C++/C/Python 1.0 APIs will be discontinued with 2024.0. 
  • PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF).

You can find OpenVINO™ toolkit 2023.2 release here:

• Download archives* with OpenVINO™
• Install it via Conda: conda install -c conda-forge openvino=2023.2.0
• OpenVINO™ for Python: pip install openvino==2023.2.0

Acknowledgements

Thanks for contributions from the OpenVINO developer community: @siddhant-0707, @NsdHSO, @mahimairaja, @SANTHOSH-MAMIDISETTI, @rsato10, @PRATHAM-SPS

Release documentation is available here: https://docs.openvino.ai/2023.2 Release Notes are available here: https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino/2023-2.html