Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Organize CLI arguments into groups #66

Open
djghosh13 opened this issue Jan 13, 2023 · 0 comments
Open

Organize CLI arguments into groups #66

djghosh13 opened this issue Jan 13, 2023 · 0 comments
Labels

Comments

@djghosh13
Copy link
Contributor

Use parser.add_argument_group() to organize the CLI arguments logically; currently, the help text is just a wall of text, and it's likely to only get longer over time. (Not a high priority though)

options:
  -h, --help            show this help message and exit
  --dataset DATASET     Dataset to use for the benchmark
  --split SPLIT         Dataset split to use
  --model MODEL         Model architecture to use from OpenCLIP
  --pretrained PRETRAINED
                        Model checkpoint name to use from OpenCLIP
  --task {zeroshot_classification,zeroshot_retrieval,linear_probe}
  --amp                 whether to use mixed precision
  --num_workers NUM_WORKERS
  --recall_k RECALL_K [RECALL_K ...]
                        for retrieval, select the k for Recall@K metric.
  --fewshot_k FEWSHOT_K
                        for linear probe, how many shots. -1 = whole dataset.
  --fewshot_epochs FEWSHOT_EPOCHS
                        for linear probe, how many epochs.
  --fewshot_lr FEWSHOT_LR
                        for linear probe, what is the learning rate.
  --skip_load           for linear probes, when everything is cached, no need
                        to load model.
  --seed SEED           random seed.
  --batch_size BATCH_SIZE
  --model_cache_dir MODEL_CACHE_DIR
                        directory to where downloaded models are cached
  --dataset_root DATASET_ROOT
                        dataset root folder where the datasets are downloaded.
  --feature_root FEATURE_ROOT
                        feature root folder where the features are stored.
  --annotation_file ANNOTATION_FILE
                        text annotation file for retrieval datasets. Only
                        needed for when `--task` is `zeroshot_retrieval`.
  --language LANGUAGE   language of classname and prompts to use for zeroshot
                        classification.
  --output OUTPUT       output file where to dump the metrics
  --verbose             verbose mode
  --cupl                Use natural language prompt from CuPL paper
  --save_clf SAVE_CLF   optionally save the classification layer output by the
                        text tower
  --load_clfs LOAD_CLFS [LOAD_CLFS ...]
                        optionally load and average mutliple layers output by
                        text towers.
@rom1504 rom1504 added the UX label Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants