This is the legacy GPT-NeoX relying on old DeeperSpeed (0.3.15). We only recommend using this release under circumstance that you're loading a model based on old DeeperSpeed (e.g. GPT-J, GPT-NeoX20B, the Pythia suite, etc).

The primary difference between this release and v2.x is the DeepSpeed version supported. If you're using 2.x, we're assuming that you're using either the latest release of DeepSpeed or DeeperSpeed 2.x.