v0.3.0 (2022-08-17)

Full Changelog

RayService (new feature!)

• [rayservice] Fix config names to match serve config format directly (#464, @edoakes)
• Disable pin on head for serve controller by default in service operator (#457, @iycheng)
• add wget timeout to probes (#448, @wilsonwang371)
• Disable async serve handler in Ray Service cluster. (#447, @iycheng)
• Add more env for RayService head or worker pods (#439, @brucez-anyscale)
• RayCluster created by RayService set death info env for ray container (#419, @brucez-anyscale)
• Add integration test for kuberay ray service and improve ray service operator (#415, @brucez-anyscale)
• Fix a potential reconcile issue for RayService and allow config unhealth time threshold in CR (#384, @brucez-anyscale)
• [Serve] Unify logger and add user facing events (#378, @simon-mo)
• Improve RayService Operator logic to handle head node crash (#376, @brucez-anyscale)
• Add serving service for users traffic with health check (#367, @brucez-anyscale)
• Create a service for dashboard agent (#324, @brucez-anyscale)
• Update RayService CR to integrate with Ray Nightly (#322, @brucez-anyscale)
• RayService: zero downtime update and healthcheck HA recovery (#307, @brucez-anyscale)
• RayService: Dev RayService CR and Controller logic (#287, @brucez-anyscale)
• KubeRay: kubebuilder creat RayService Controller and CR (#270, @brucez-anyscale)

RayJob (new feature!)

• Properly convert unix time into meta time (#480, @pingsutw)
• Fix nil pointer dereference (#429, @pingsutw)
• Improve RayJob controller quality to alpha (#398, @Jeffwan)
• Submit ray job after cluster is ready (#405, @pingsutw)
• Add RayJob CRD and controller logic (#303, @harryge00)

Cluster Fault Tolerant (new feature!)

• tune readiness probe timeouts (#411, @wilsonwang371)
• enable ray external storage namespace (#406, @wilsonwang371)
• Initial support for external Redis and GCS HA (#294, @wilsonwang371)

Autoscaler (new feature!)

• [Autoscaler] Match autoscaler image to Ray head image for Ray >= 2.0.0 (#423, @DmitriGekhtman)
• [autoscaler] Better defaults and config options (#414, @DmitriGekhtman)
• [autoscaler] Make log file mount path more specific. (#391, @DmitriGekhtman)
• [autoscaler] Flip prioritize-workers-to-delete feature flag (#379, @DmitriGekhtman)
• Update autoscaler image (#371, @DmitriGekhtman)
• [minor] Update autoscaler image. (#313, @DmitriGekhtman)
• Provide override for autoscaler image pull policy. (#297, @DmitriGekhtman)
• [RFC][autoscaler] Add autoscaler container overrides and config options for scale behavior. (#278, @DmitriGekhtman)
• [autoscaler] Improve autoscaler auto-configuration, upstream recent improvements to Kuberay NodeProvider (#274, @DmitriGekhtman)

Operator

• correct gcs ha to gcs ft (#482, @wilsonwang371)
• Fix panic in cleanupInvalidVolumeMounts (#481, @MissiontoMars)
• fix: worker node can't connect to head node service (#445, @pingsutw)
• Add http resp code check for kuberay (#435, @brucez-anyscale)
• Fix wrong ray start command (#431, @pingsutw)
• fix controller: use Service's TargetPort (#383, @davidxia)
• Generate clientset for new specs (#392, @Basasuya)
• Add Ray address env. (#388, @DmitriGekhtman)
• Add the support to replace evicted head pod (#381, @Jeffwan)
• [Bug] Fix raycluster updatestatus list wrong label (#377, @scarlet25151)
• Make replicas optional for the head spec. (#362, @DmitriGekhtman)
• Add ray head service endpoints in status for expose raycluster's head node endpoints (#341, @scarlet25151)
• Support KubeRay management labels (#345, @Jeffwan)
• fix: bug in object store memory validation (#332, @davidxia)
• feat: add EventReason type for events (#334, @davidxia)
• minor refactor: fix camel-casing of unHealthy -> unhealthy (#333, @davidxia)
• refactor: remove redundant imports (#317, @davidxia)
• Fix GPU-autofill for rayStartParams (#328, @DmitriGekhtman)
• ray-operator: add missing space in controller log messages (#316, @davidxia)
• fix: use head group's ServiceAccount in autoscaler RoleBinding (#315, @davidxia)
• fix typos in comments and help messages (#304, @davidxia)
• enable force cluster upgrade (#231, @wilsonwang371)
• fix operator: correctly set head pod service account (#276, @davidxia)
• [hotfix] Fix Service account typo (#285, @DmitriGekhtman)
• Rename RayCluster folder to Ray since the group is Ray (#275, @brucez-anyscale)
• KubeRay: Relocate files to enable controller extension with Kubebuilder (#268, @brucez-anyscale)
• fix: use configured RayCluster service account when autoscaling (#259, @davidxia)
• suppress not found errors into regular logs (#222, @akanso)
• adding label check (#221, @akanso)
• Prioritize WorkersToDelete (#208, @sriram-anyscale)
• Simplify k8s client creation (#179, @chenk008)
• [ray-operator]Make log timestamp readable (#206, @chenk008)
• bump controller-runtime to 0.11.1 and Kubernetes to v1.23 (#180, @chenk008)

APIServer

• Add envs in cluster service api (#432, @MissiontoMars)
• Expose swallowed detail error messages (#422, @Jeffwan)
• fix: typo RAY_DISABLE_DOCKER_CPU_WRARNING -> RAY_DISABLE_DOCKER_CPU_WARNING (#421, @pingsutw)
• Add hostPathType and mountPropagationMode field for apiserver (#413, @scarlet25151)
• Fix ListAllComputeTemplates proto comments (#407, @MissiontoMars)
• Enable DefaultHTTPErrorHandler and Upgrade grpc-gateway to v2 (#369, @Jeffwan)
• Validate namespace consistency in the request when creating the cluster and the compute template (#365, @daikeshi)
• Update compute template service url to include namespace path param (#363, @Jeffwan)
• fix apiserver created raycluster metrics port missing and check (#356, @scarlet25151)
• Support mounting volumes in API request (#346, @Jeffwan)
• add standard label for the filtering of cluster (#342, @scarlet25151)
• expose kubernetes events in apiserver (#343, @scarlet25151)
• Update ray-operator version in the apiserver (#340, @pingsutw)
• fix: typo worker_group_sepc -> worker_group_spec (#330, @davidxia)
• Fix gpu-accelerator in template (#296, @armandpicard)
• Add namespace scope to compute template operations (#244, @daikeshi)
• Add namespace scope to list operation (#237, @daikeshi)
• Add namespace scope for Ray cluster get and delete operations (#229, @daikeshi)

CLI

• Cli: make namespace optional to adapt to ListAll operation (#361, @Jeffwan)

Deployment (kubernetes & helm)

• sync up helm chart's role (#472, @scarlet25151)
• helm-charts/ray-cluster: Allow extra workers (#451, @ulfox)
• Update helm chart version to 0.3.0 (#461, @Jeffwan)
• helm-chart/ray-cluster: allow head autoscaling (#443, @ulfox)
• modify kuberay operator crds in kuberay operator chart and add apiserver chart (#354, @scarlet25151)
• Warn explicitly against using kubectl apply to create RayCluster CRD. (#302, @DmitriGekhtman)
• Sync crds to Helm chart (#280, @haoxins)
• [Feature]Run kuberay in a single namespace (#258, @wilsonwang371)
• fix duplicated port config and manager.yaml missing config (#250, @wilsonwang371)
• manifests: Add live/ready probes (#243, @haoxins)
• Helm: supports custom probe seconds (#239, @haoxins)
• Add CD for helm charts (#199, @ddelange)

Build and Testing

• Enable docker image push for release-0.3 branch (#462, @Jeffwan)
• add new 8000 port forwarding in kind (#424, @wilsonwang371)
• improve compatibility test stability (#418, @wilsonwang371)
• improve test stability (#394, @wilsonwang371)
• use more strict formatting (#385, @wilsonwang371)
• fix flaky test issue (#370, @wilsonwang371)
• provide more detailed information in case of test failures (#352, @wilsonwang371)
• fix wrong kuberay image used by compatibility test (#327, @wilsonwang371)
• add cluster nodes info test (#299, @wilsonwang371)
• Fix the image name in deploy cmd (#293, @brucez-anyscale)
• [CI]enable ci test to check ctrl plane health state (#279, @wilsonwang371)
• [bugfix]update flaky test timeout (#254, @wilsonwang371)
• Update format by running gofumpt (#236, @wilsonwang371)
• Add unit tests for raycluster_controller reconcilePods function (#219, @Waynegates)
• Support ray 1.12 (#245, @wilsonwang371)
• add 1.11 to compatibility test and update comment (#217, @wilsonwang371)
• run compatibility in parallel using multiple workflows (#215, @wilsonwang371)

Monitoring

• add-state-machine-and-exposing-port (#319, @scarlet25151)
• Install: Fix directory path for prometheus install.sh (#256, @Tomcli)
• Fix Ray Operator prometheus config (#253, @Tomcli)
• Emit prometheus metrics from kuberay control plane (#232, @Jeffwan)
• Enable metrics-export-port by default and configure prometheus monitoring (#230, @scarlet25151)

Docs

• [doc] Config and doc updates ahead of KubeRay 0.3.0/Ray 2.0.0 (#486, @DmitriGekhtman)
• document the raycluster status (#473, @scarlet25151)
• Clean up example samples (#434, @DmitriGekhtman)
• Add ray state api doc link in ray service doc (#428, @brucez-anyscale)
• [docs] Add sample configs with larger Ray pods (#426, @DmitriGekhtman)
• Add RayJob docs and development docs (#404, @Jeffwan)
• Add gcs ha doc into mkdocs (#402, @brucez-anyscale)
• [minor] Add client and dashboard ports to ports in example configs. (#399, @DmitriGekhtman)
• Add documentation for RayService (#387, @brucez-anyscale)
• Fix broken links by creating referenced soft links (#335, @Jeffwan)
• Support hosting swagger ui in apiserver (#344, @Jeffwan)
• Remove autoscaler debug example to prevent confusion (#326, @Jeffwan)
• Add a link to protobuf-grpc-service design page in proto doc (#310, @yabuchan)
• update readme and address issue #286 (#311, @wilsonwang371)
• docs fix: specify only Go 1.16 or 1.17 works right now (#261, @davidxia)
• Add documention link in readme (#247, @simon-mo)
• Use mhausenblas/mkdocs-deploy-gh-pages action for docs (#233, @Jeffwan)
• Build KubeRay Github site (#216, @Jeffwan)

Contributors

Thank you to everyone who contributed to this release! ❤️

Users whose commits are in this release (alphabetically by user name) @akanso @armandpicard @Basasuya @brucez-anyscale @chenk008 @daikeshi @davidxia @ddelange @DmitriGekhtman @edoakes @haoxins @harryge00 @iycheng @Jeffwan @MissiontoMars @pingsutw @scarlet25151 @simon-mo @sriram-anyscale @Tomcli @ulfox @Waynegates @wilsonwang371 @yabuchan

A special shoutout to these folks who helped report, test, and review the codes: @caitengwei @ericl @pcmoritz @wuisawesome

And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏