Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore scheduler functionality #4

Open
marrrcin opened this issue Mar 11, 2022 · 4 comments
Open

Restore scheduler functionality #4

marrrcin opened this issue Mar 11, 2022 · 4 comments
Labels
enhancement New feature or request

Comments

@marrrcin
Copy link
Contributor

Scheduler was temporarily disabled due to slight issues with MLFlow integrations (token obtaining) and not working re-scheduling (old schedules were not disabled).
Restore this functionality in future releases.

@em-pe em-pe added the enhancement New feature or request label Mar 24, 2022
@Lasica
Copy link
Contributor

Lasica commented Nov 22, 2023

Is this still relevant? I don't understand the context of this issue.

@Stormz4
Copy link

Stormz4 commented Jan 24, 2024

Are the any news about this issue?

@marrrcin
Copy link
Contributor Author

We could potentially use https://cloud.google.com/vertex-ai/docs/pipelines/schedule-pipeline-run#aiplatform_create_pipeline_schedule_sample-python_vertex_ai_sdk 🤔

Let me take it to the team and we'll get back to you :)

@Stormz4
Copy link

Stormz4 commented Jan 26, 2024

I solved this problem with the following workaround:

  1. Compile the Kedro project with kedro vertexai compile
  2. Then, through a python script:
    1. Create a PipelineJob object
      (https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform.PipelineJob)
    2. Call the function create_schedule of said PipelineJob (https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform.PipelineJob#google_cloud_aiplatform_PipelineJob_create_schedule)

It should be pretty streamlined to implement said solution in Kedro's API.

optional step suggestion (could be a flag inserted through args):

  • Check if a job already exists with my desired display name: if it exists, delete it (and recreate it). This is needed since in vertexai you can have multiple pipeline with the same name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants