Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

8 fine tuning #14

Merged
merged 101 commits into from
May 28, 2024
Merged

8 fine tuning #14

merged 101 commits into from
May 28, 2024

Commits on Apr 24, 2024

  1. model and tokenizer loading

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    bcb9b7b View commit details
    Browse the repository at this point in the history

  2. additional dependencies

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    2b6ba63 View commit details
    Browse the repository at this point in the history

  3. configs, training script, generation

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    3f69c81 View commit details
    Browse the repository at this point in the history

  4. removed generated configs accidentally commited

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    2cfed79 View commit details
    Browse the repository at this point in the history

  5. train args

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    a9e5028 View commit details
    Browse the repository at this point in the history

  6. early bask script generation

    @philswatton @jack89roberts
    philswatton authored and jack89roberts committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    38562f3 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. removed depracted property

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    dac3fe5 View commit details
    Browse the repository at this point in the history

  2. added wandb, broke toplevel config in process

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    a331b61 View commit details
    Browse the repository at this point in the history

  3. placeholder preprocessing

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    c994e07 View commit details
    Browse the repository at this point in the history

  4. dependencies

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    6180a99 View commit details
    Browse the repository at this point in the history

  5. example tofu configs

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    7e11b51 View commit details
    Browse the repository at this point in the history

  6. added placeholders for 1st debug run, almost ready to run

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    0b8fb48 View commit details
    Browse the repository at this point in the history

  7. placeholder data config loading

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    88c8735 View commit details
    Browse the repository at this point in the history

  8. added remaining required placeholders

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    f824f8c View commit details
    Browse the repository at this point in the history

  9. fixed mistake in copying wandb config

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    662799d View commit details
    Browse the repository at this point in the history

  10. to dict method started

    @philswatton
    philswatton committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    05b9d0f View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. fixed import error

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    290ccfa View commit details
    Browse the repository at this point in the history

  2. fixed name of from_yaml

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    d3aecc2 View commit details
    Browse the repository at this point in the history

  3. fixed path typo

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    3a719d7 View commit details
    Browse the repository at this point in the history

  4. fixed kwarg error

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    dc84fda View commit details
    Browse the repository at this point in the history

  5. fixed misaligned class and config names

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    4525606 View commit details
    Browse the repository at this point in the history

  6. added seed kwarg

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    b35ad39 View commit details
    Browse the repository at this point in the history

  7. added vscode ignore

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    2338796 View commit details
    Browse the repository at this point in the history

  8. bugs for days

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    ba55154 View commit details
    Browse the repository at this point in the history

  9. kill me now

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    20fe114 View commit details
    Browse the repository at this point in the history

  10. last fix

    @philswatton
    philswatton committed May 1, 2024
    Configuration menu
    Copy the full SHA
    6cbfc6c View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. fixed str learning rate

    @philswatton
    philswatton committed May 3, 2024
    Configuration menu
    Copy the full SHA
    0a76af6 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. 🚧 tmp - quick way to ensure unique save dir

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    7684f31 View commit details
    Browse the repository at this point in the history

  2. 🔧 100 epoch, default LR, gpt2 job for comparison

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    8d51b6a View commit details
    Browse the repository at this point in the history

  3. 🔧 add missing slurm script from previous commit

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    527c90d View commit details
    Browse the repository at this point in the history

  4. 🔧 reduce walltime

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    4150b78 View commit details
    Browse the repository at this point in the history

  5. 🔧 trainingargs doesn't like null logging steps

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    d3d5896 View commit details
    Browse the repository at this point in the history

  6. 🔧 save strategy to epoch too...

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    d1eea19 View commit details
    Browse the repository at this point in the history

  7. remove colon from placeholder preprocessing 

    As formatting questions like "Who is Bob?: Bob is an astronaut." seems a bit odd.
    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    97b13ed View commit details
    Browse the repository at this point in the history

  8. 🦺 only save max 2 checkpoints per experiment

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    cd69dbf View commit details
    Browse the repository at this point in the history

  9. wip train on all data

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    ae1d12d View commit details
    Browse the repository at this point in the history

  10. train on retain or all quick add

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    5286c55 View commit details
    Browse the repository at this point in the history

  11. update submit scripts

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    f37cfd2 View commit details
    Browse the repository at this point in the history

  12. walltime

    @jack89roberts
    jack89roberts committed May 7, 2024
    Configuration menu
    Copy the full SHA
    6ca1943 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. trainer kwargs kept as dict

    @philswatton
    philswatton committed May 9, 2024
    Configuration menu
    Copy the full SHA
    3c42751 View commit details
    Browse the repository at this point in the history

  2. wandb arg to trainer

    @philswatton
    philswatton committed May 9, 2024
    Configuration menu
    Copy the full SHA
    0ade9e2 View commit details
    Browse the repository at this point in the history

  3. updated kwarg processing

    @philswatton
    philswatton committed May 9, 2024
    Configuration menu
    Copy the full SHA
    5771b35 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. full arg names in configs

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    f95422a View commit details
    Browse the repository at this point in the history

  2. removed wandb arg in model configs

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    d5b40fe View commit details
    Browse the repository at this point in the history

  3. more config updates

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9f026b5 View commit details
    Browse the repository at this point in the history

  4. added optional second dataset to training, removed wandb from model c… 

    …onfig cls
    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    fab38b2 View commit details
    Browse the repository at this point in the history

  5. report to none as intended

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    6e32a29 View commit details
    Browse the repository at this point in the history

  6. seed in trainer, trainer class now determined by string

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    73253a8 View commit details
    Browse the repository at this point in the history

  7. early stopping now optional

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    cd91403 View commit details
    Browse the repository at this point in the history

  8. steps can already be floats

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    73eb67d View commit details
    Browse the repository at this point in the history

  9. absolute project path added

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    8bf73c8 View commit details
    Browse the repository at this point in the history

  10. now using absolute paths (except for output, which is still TODO)

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    1351110 View commit details
    Browse the repository at this point in the history

  11. file renames

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    e0292c4 View commit details
    Browse the repository at this point in the history

  12. renamed imports

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    fb44636 View commit details
    Browse the repository at this point in the history

  13. removed easier imports

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    5e71095 View commit details
    Browse the repository at this point in the history

  14. experiment config to_dict

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    8c6c13c View commit details
    Browse the repository at this point in the history

  15. moved constants

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    f4159c9 View commit details
    Browse the repository at this point in the history

  16. adding padding token now optional

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    aa13b32 View commit details
    Browse the repository at this point in the history

  17. separate hyperparameters config

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    90b6590 View commit details
    Browse the repository at this point in the history

  18. enabled LoRA (in theory)

    @philswatton
    philswatton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    8863c1b View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. data config

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    17e50fc View commit details
    Browse the repository at this point in the history

  2. resolved tofu.py merge conflict

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    e80f19d View commit details
    Browse the repository at this point in the history

  3. data config added to experiment config

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    cd59b3e View commit details
    Browse the repository at this point in the history

  4. slightly changing configs

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    45662a7 View commit details
    Browse the repository at this point in the history

  5. rename

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    fd2b1ef View commit details
    Browse the repository at this point in the history

  6. renamed yaml file to be used

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    5a4ce10 View commit details
    Browse the repository at this point in the history

  7. paths

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    25d6e6d View commit details
    Browse the repository at this point in the history

  8. docstrings on base and experiment configs

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    d37aa8e View commit details
    Browse the repository at this point in the history

  9. docstrings

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    c783e21 View commit details
    Browse the repository at this point in the history

  10. removed placeholder comments

    @philswatton
    philswatton committed May 16, 2024
    Configuration menu
    Copy the full SHA
    c78d3c3 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. top config generation

    @philswatton
    philswatton committed May 20, 2024
    Configuration menu
    Copy the full SHA
    194ee62 View commit details
    Browse the repository at this point in the history

  2. some extra ignores

    @philswatton
    philswatton committed May 20, 2024
    Configuration menu
    Copy the full SHA
    f9bfec2 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. tweak to config cls

    @philswatton
    philswatton committed May 21, 2024
    Configuration menu
    Copy the full SHA
    b6e88ae View commit details
    Browse the repository at this point in the history

  2. config tests

    @philswatton
    philswatton committed May 21, 2024
    Configuration menu
    Copy the full SHA
    2a62e2b View commit details
    Browse the repository at this point in the history

  3. more useful gitignore

    @philswatton
    philswatton committed May 21, 2024
    Configuration menu
    Copy the full SHA
    f5aa067 View commit details
    Browse the repository at this point in the history

  4. finished up config init tests

    @philswatton
    philswatton committed May 21, 2024
    Configuration menu
    Copy the full SHA
    146cd0d View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. not testing these - just calls to other fns

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    95f0943 View commit details
    Browse the repository at this point in the history

  2. integrated preprocessing PR

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    2730763 View commit details
    Browse the repository at this point in the history

  3. top config now uses array job submission

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    4fbf9cf View commit details
    Browse the repository at this point in the history

  4. debug top config:

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    7427f13 View commit details
    Browse the repository at this point in the history

  5. more debugging

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    1cd3c23 View commit details
    Browse the repository at this point in the history

  6. fixed data configs

    @philswatton
    philswatton committed May 22, 2024
    Configuration menu
    Copy the full SHA
    8bd22f4 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. updated data cfg test

    @philswatton
    philswatton committed May 23, 2024
    Configuration menu
    Copy the full SHA
    a9d477c View commit details
    Browse the repository at this point in the history

  2. bug fixing

    @philswatton
    philswatton committed May 23, 2024
    Configuration menu
    Copy the full SHA
    d429300 View commit details
    Browse the repository at this point in the history

  3. finished README sections for PR

    @philswatton
    philswatton committed May 23, 2024
    Configuration menu
    Copy the full SHA
    458f5c1 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Update src/arcsf/models/model.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    6c86740 View commit details
    Browse the repository at this point in the history

  2. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    068e47f View commit details
    Browse the repository at this point in the history

  3. removed cpu per gpu

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    f775085 View commit details
    Browse the repository at this point in the history

  4. fixed error from treating str as path

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    0c3878c View commit details
    Browse the repository at this point in the history

  5. updated argparse docstring

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    c4b6df3 View commit details
    Browse the repository at this point in the history

  6. short fn names + args to single line

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    8c6cf85 View commit details
    Browse the repository at this point in the history

  7. docstrings for top config fns

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    455f13b View commit details
    Browse the repository at this point in the history

  8. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    033d434 View commit details
    Browse the repository at this point in the history

  9. copy -> deepcopy

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    dbb7184 View commit details
    Browse the repository at this point in the history

  10. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    dafea7e View commit details
    Browse the repository at this point in the history

  11. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    e6c9bd3 View commit details
    Browse the repository at this point in the history

  12. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    76dd37d View commit details
    Browse the repository at this point in the history

  13. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    bba0633 View commit details
    Browse the repository at this point in the history

  14. Update src/arcsf/config/experiment.py 

    Co-authored-by: Jack Roberts <[email protected]>
    @philswatton @jack89roberts
    philswatton and jack89roberts authored May 28, 2024
    Configuration menu
    Copy the full SHA
    512eadc View commit details
    Browse the repository at this point in the history

  15. rename fn

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    d774c9b View commit details
    Browse the repository at this point in the history

  16. refactored top config generation

    @philswatton
    philswatton committed May 28, 2024
    Configuration menu
    Copy the full SHA
    a576e96 View commit details
    Browse the repository at this point in the history