WIP Model card #53

merveenoyan · 2022-12-03T14:55:15Z

This is a WIP for model card. Opening this PR so you could take a quick look to see if the behavior seems good.
I will add what I can add on top of it (Evaluation results, model plot and so on)

merveenoyan · 2022-12-03T14:55:43Z

I checked out from my documentation PR, feel free to ignore the parts about adding docstrings.

sayakpaul

Looking good so far.

Did you use the model card utility as a part of the pipeline yet?

No worries, if not. I was just being curious.

training_pipeline/pipeline/components/HFPusher/model_card.py

sayakpaul · 2022-12-03T16:05:12Z

training_pipeline/pipeline/components/HFPusher/model_card.py

+        model_metadata (dict): Dict of card metadata.
+        see here for what you can pass to the metadata section:


Include descriptions about the other arguments too.

sayakpaul · 2022-12-03T16:06:26Z

training_pipeline/pipeline/components/HFPusher/model_card.py

+    """
+    if model_metadata is None:
+        model_metadata = {}
+    model_metadata["library_name"] = "vertex"


If one runs the pipeline on a different orchestrator then vertex won't make sense. Also, it's better to call the key orchestrator or runner to denote that it's the platform on which things are getting executed.

sayakpaul · 2022-12-03T16:06:48Z

training_pipeline/pipeline/components/HFPusher/model_card_template.md

@@ -0,0 +1,27 @@
+---
+# For reference on model card metadata, see: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1


see => refer.

sayakpaul · 2022-12-03T16:38:30Z

training_pipeline/pipeline/components/HFPusher/model_card_template.md

+
+## Model Description
+
+This is a {{ task | default("segmentation")}} model trained using Vertex AI.


If we're not using Vertex AI, then it doesn't make sense.

What is this project specifically made for?

The project can be run any compatible platform. We ran it locally and Vertex AI.

Cc: @deep-diver

yes it is. the pipeline has two targets, Local and Vertex AI (which is Kubeflow 2.x). Better to say something like model trained using TensorFlow Extended

sayakpaul · 2022-12-03T16:39:03Z

training_pipeline/pipeline/components/HFPusher/model_card_template.md

@@ -0,0 +1,27 @@
+---
+# For reference on model card metadata, see: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1


Does this link also show how we can specify the formatting like:

This is a {{ task | default("segmentation")}} model trained using Vertex AI.

it just shows the keys, so nope.
it's a jinja template, I can add that as a note.

Co-authored-by: Sayak Paul <[email protected]>

deep-diver · 2022-12-05T23:38:40Z

It looks good as a starting point. I just want to ask to add a notebook that shows before and after running create_card() function to clearly see what the generated model card looks like. WDYT?

merveenoyan · 2022-12-07T09:09:47Z

@deep-diver sure I'll do that 🙂

deep-diver · 2022-12-07T12:40:05Z

@merveenoyan thanks! that would be great 👍🏼

sayakpaul · 2023-01-16T12:37:33Z

@deep-diver

@merveenoyan is finishing the first version of the model card feature in this PR. She wanted to test HFPusher (which has the model card utility) in isolation, preferably in a notebook.

Do you have any starter notebook which she could use?

merveenoyan · 2023-01-16T15:09:45Z

The revision of the test repository lies here, I tested like below (thought it would be good to test deployment in isolation but there might be a problem with HfPusher calling this, I haven't test separately yet) https://huggingface.co/merve/tfx-blessed-model/blob/merve/README.md
@sayakpaul @deep-diver

import tensorflow as tf
from runner import deploy_model_for_hf_hub

model = model_init() # random dummy model
model.save("./model_save_dir")

model_card_metadata = {}
deploy_model_for_hf_hub(
    username="merve",
    access_token=access_token,
    repo_name="tfx-blessed-model",
    model_path="./model_save_dir",
    model_version="merve"
)

merveenoyan · 2023-01-16T15:15:31Z

@deep-diver had to add create_card to __init__ so it was importable :/

sayakpaul

Thanks a lot for working on this!

Will defer to @deep-diver for the final merge.

merveenoyan · 2023-01-17T14:13:56Z

Closes #43

deep-diver · 2023-01-17T14:16:07Z

Let's merge this PR as a starting point. @merveenoyan @sayakpaul

There is a couple of things to mention/check:

It looks like Hugging Face Model repo only shows model card in Model Card Tab in the main branch. Am I right? Like even though I switch to a certain branch.
I see that you create model_card_template.md file in Python code. I think it is better if we could keep the template as a static file in the project repo. I have recently upgraded HFPusher to push additional files to Model repository (Link). In this reference, I did it to include custom_handler.py that refers to the latest branch of a Model repo. Let's see if we could use this feature to include model_card_template.md in the future. It could automatically inject HF username or some additional metadata.
- this feature is not merged in the TFX Addon yet though.

merveenoyan added 2 commits December 3, 2022 16:41

documentation and readability nits

fe3d187

initial commit

fcf44aa

merveenoyan requested a review from deep-diver December 3, 2022 14:55

merveenoyan requested a review from sayakpaul December 3, 2022 14:55

sayakpaul requested changes Dec 3, 2022

View reviewed changes

Update training_pipeline/pipeline/components/HFPusher/model_card.py

d7c679b

Co-authored-by: Sayak Paul <[email protected]>

addressed comments

2de9d3c

merveenoyan marked this pull request as ready for review January 16, 2023 12:28

fixes

f2d7f4a

add create_card to __init__

654d149

merveenoyan added 2 commits January 16, 2023 16:20

revert

bd97496

absolute import

868e5b9

merveenoyan requested a review from sayakpaul January 16, 2023 15:29

sayakpaul approved these changes Jan 17, 2023

View reviewed changes

deep-diver merged commit 2f95ae1 into deep-diver:main Jan 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP Model card #53

WIP Model card #53

merveenoyan commented Dec 3, 2022 •

edited

Loading

merveenoyan commented Dec 3, 2022

sayakpaul left a comment

sayakpaul Dec 3, 2022

sayakpaul Dec 3, 2022

sayakpaul Dec 3, 2022

merveenoyan Dec 4, 2022

sayakpaul Dec 3, 2022

merveenoyan Dec 4, 2022

sayakpaul Dec 4, 2022

deep-diver Dec 5, 2022 •

edited

Loading

sayakpaul Dec 3, 2022

merveenoyan Dec 4, 2022 •

edited

Loading

deep-diver commented Dec 5, 2022

merveenoyan commented Dec 7, 2022

deep-diver commented Dec 7, 2022

sayakpaul commented Jan 16, 2023

merveenoyan commented Jan 16, 2023 •

edited

Loading

merveenoyan commented Jan 16, 2023

sayakpaul left a comment

merveenoyan commented Jan 17, 2023

deep-diver commented Jan 17, 2023

		model_metadata (dict): Dict of card metadata.
		see here for what you can pass to the metadata section:

		@@ -0,0 +1,27 @@
		---
		# For reference on model card metadata, see: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1


		## Model Description

		This is a {{ task \| default("segmentation")}} model trained using Vertex AI.

WIP Model card #53

WIP Model card #53

Conversation

merveenoyan commented Dec 3, 2022 • edited Loading

merveenoyan commented Dec 3, 2022

sayakpaul left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deep-diver Dec 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merveenoyan Dec 4, 2022 • edited Loading

Choose a reason for hiding this comment

deep-diver commented Dec 5, 2022

merveenoyan commented Dec 7, 2022

deep-diver commented Dec 7, 2022

sayakpaul commented Jan 16, 2023

merveenoyan commented Jan 16, 2023 • edited Loading

merveenoyan commented Jan 16, 2023

sayakpaul left a comment

Choose a reason for hiding this comment

merveenoyan commented Jan 17, 2023

deep-diver commented Jan 17, 2023

merveenoyan commented Dec 3, 2022 •

edited

Loading

deep-diver Dec 5, 2022 •

edited

Loading

merveenoyan Dec 4, 2022 •

edited

Loading

merveenoyan commented Jan 16, 2023 •

edited

Loading