Data WG roadmap for KF 1.10

[rimolive]

Hello @kubeflow/wg-data-leads @tarilabs @ChenYi015

The 1.10 release team will use this tracking issue to coordinate the integration of the Model Registry and Spark Operator with the Kubeflow 1.10 release.

Heads up that the feature freeze is scheduled for January 20th 2025 (needs confirmation). For the full release schedule, refer to the 1.10 release timeline.

(note: links are pending due to #761)

To prepare for the 1.10 release, please provide answers to the following questions:

1. What version of Model Registry and Spark Operator would you like to include for the 1.10 release?
2. Could you provide a branch/tag for this version? (note: It doesn't have to be final)
3. Can you share a link to all open / work in progress issues that you're expecting to include in the 1.10?
4. If you're using the following dependencies, could you provide the versions you'll be supporting for each dependency?

• KServe/KNative
• Istio
• Kubernetes
• Cert-Manager
• Kustomize

The release team WG liaison for Pipelines WG will be @milosjava and you can keep track of high-level release updates in the #768 issue.

[tarilabs]

from Model Registry WG

• theme: Kubeflow Model Registry UI
• roadmap/details: [Kubeflow 1.10] Define Kubeflow 1.10 Roadmap for Model Registry model-registry#175

cc/ @milosjava @rareddy @ederign

[rimolive]

From Spark Operator:

• roadmap: Spark Operator Roadmap 2024 spark-operator#2193

[juliusvonkohout]

We need a maintainer for the spark manifests at https://github.com/kubeflow/manifests/tree/master/contrib/spark. See also https://github.com/kubeflow/manifests/blob/master/proposals/20220926-contrib-component-guidelines.md#component-requirements for PSS restricted requirements and there should be a synchronization script in /hack even if it just resembles the current makefile

[rimolive]

/assign @milosjava

[varodrig]

not using a specific version, usiing from platform wg

[varodrig]

We need a maintainer for the spark manifests at https://github.com/kubeflow/manifests/tree/master/contrib/spark. See also https://github.com/kubeflow/manifests/blob/master/proposals/20220926-contrib-component-guidelines.md#component-requirements for PSS restricted requirements and there should be a synchronization script in /hack even if it just resembles the current makefile

@milosjava to follow up on this

[milosjava]

Model registry , no red flags , blockers….
Progressing well on the development. What is still pending to be 'ready' is the integration with Kubeflow Central Dashboard and also how we are doing to deploy our components in the manifests repo... but this is already in progress

[milosjava]

Model Registry no blockers, we can say that approx 50% is done. As Josh suggested on a community call we will find an end-user and go through docs together. My idea for the end user is one of our (pepsico) projects.

Regarding SparkOperator and the follow up:

• Might need to create a Kustomize manifest for the operator sincemost users and the dev work that was done has been on the Helm chart
• The doc work can be done
• The existing end to end tests should be fine for the testing but one could be pulled out as a specific smoke test
• Some work on the security stance is done but need to follow up with Julius for some other changes

To summarize: Smoke test, security stance, Kustomize and doc work are doable. Need more info on Integration between the operator and Kubeflow
Here is the issue to track If that is fixed we can include it in Kubeflow 1.10

[milosjava]

Also I think would be possible to find end-user for SparkOperator as well.

[juliusvonkohout]

@milosjava

Spark-operstor is already in our CICD. Here are the remaining issues to promote it to an application kubeflow/manifests#2912

[milosjava]

@rimolive Sharing model registry progress track: https://github.com/kubeflow/model-registry/milestones

[milosjava]

UI road map : kubeflow/model-registry#175 (comment)

[tarilabs]

from Milos in Release meeting:

• Model Registry tracker and fine-grainer tracker for UI per Data WG roadmap for KF 1.10 #769 (comment)
• spark operator is OK but there is some integration work ongoing for Kubeflow

[varodrig]

We need to follow up on this @milosjava any updates

[milosjava]

Model Registry, still in green, no blockers. current focus on kubeflow/model-registry#599

[tarilabs]

@milosjava mentions in the Release meeting:

• MR progressing there is a call following the Release meeting
• Spark operator did not respond to request for feedback/update (monthly call is a bit TZ difficult)

[milosjava]

Model Registry progressing, no issues.

Spark Operator: seems there is an issue with integration with Kubeflow. I am in the contact with number of Spark Operator developers and it seems istio support is problem and probably we need help with this. More info: kubeflow/manifests#2912 (comment)