v0.19.0: IPEX Support, Foundations for Transformers Integration, FP8 for Ada Lovelace GPUs, and Squashed Bugs

What's New

• Support for Intel IPEX support has been added, check out the how-to guide now!
• Various modifications have been added to begin work on having 🤗 Accelerate be the foundation for the Trainer, keep an eye on the repos to see how our progress is coming along!
• FP8 training is now supported on Ada Lovelance GPUs
• The wandb integration now supports logging of images and tables through tracker.log_images and tracker.log_tables respectively
• Many, many squashed bugs! (see the full detailed report for just what they were)
• 17 new contributors to the framework, congratulations to all who took their first step! 🚀

What's Changed

• Fix pypi image by @muellerzr in #1249
• raise error when dataloader with None as batch_size when using DS by @pacman100 in #1250
• Handle missing deepspeed config by @HeyangQin in #1251
• [core] Add Quantization support for dispatch_model by @younesbelkada in #1237
• Check attribute 'overflow' exists in optimizer. by @tensimiku in #1259
• ipex intel extension for pytorch integration by @sywangyi in #1255
• fix issue template by @stas00 in #1264
• Change error raised to ValueError by @sgugger in #1267
• Fix reduce operation by @xyfJASON in #1268
• Raise import error if fp8 not available in has_transfomer_engine_layers by @muellerzr in #1283
• Add missing FP8 options to CLI by @muellerzr in #1284
• Update quicktour.mdx by @StandardAI in #1273
• Minor fix whitespace colon by @guspan-tanadi in #1272
• fix attribute error in DataloaderShared by @ZhiyuanChen in #1278
• Fix TypeError bug in honor_type by @muellerzr in #1285
• Raise more explicit error when transformer_engine isn't installed by @muellerzr in #1287
• Expound error on recursively_apply by @muellerzr in #1286
• Only check for dtype if it has it in get_state_dict by @muellerzr in #1288
• [bnb] fix bnb slow test by @younesbelkada in #1292
• Raise better error on notebook_launcher by @muellerzr in #1293
• Make note about grad accum and prec in performance documentation by @muellerzr in #1296
• fix for load_checkpoint_and_dispatch(device_map=None) by @anentropic in #1297
• Set the state device dependant to Accelerator on multigpu by @muellerzr in #1220
• add usage guide for ipex plugin by @sywangyi in #1270
• Simplify MPS implementation by @sgugger in #1308
• Bug fix in setattr by @aashiqmuhamed in #1312
• Allow xpu backend by @muellerzr in #1313
• Default to nccl by @muellerzr in #1314
• offload the previous module hook before the current module is moved to… by @williamberman in #1315
• Ensure that dynamo is compatible with mixed precision by @muellerzr in #1318
• Upgrade torch version on main tests by @muellerzr in #1323
• Add test flag and import check for dynamo by @muellerzr in #1322
• ensure module prefixes only match that module by @xloem in #1319
• remove repetitive entries from device lists by @xloem in #1321
• Fix failing test on main by @muellerzr in #1332
• Verbosity, Progress Bar for Loading by @xloem in #1329
• Skip failing torch 2.0+ test by @muellerzr in #1339
• Remove unused amp import util by @muellerzr in #1340
• Fix nested context manager for main_process_first() by @flukeskywalker in #1304
• Small progress bar fix by @xloem in #1341
• Pop more backend options by @muellerzr in #1342
• Support FP8 mixed precision training for Ada Lovelace GPUs by @Dango233 in #1348
• using deepspeed.comm for distrbiuted init by @pacman100 in #1352
• [bnb] Fix bnb slow test by @younesbelkada in #1355
• Better check for packages availability by @apbard in #1356
• fix: typing issues, and replace deprecated python typing (Optional, Union) to | by @kiyoon in #1363
• Fix default FSDP_MIN_NUM_PARAMS so it's an int by @sam-hieken in #1367
• Special transformers case from args by @muellerzr in #1364
• Improve accelerate env reporting by @muellerzr in #1376
• Seperate out contextmanager generation by @muellerzr in #1379
• delete textfile after tests are done by @muellerzr in #1381
• Fix flakey thread issue by @muellerzr in #1387
• fix config bug for 'mixed_precision' from 'yaml.safe_load()' by @ys-eric-choi in #1386
• Log Images and other types to wandb by @tcapelle in #962
• Bump torch version by @muellerzr in #1392
• Fix gather_obj by @muellerzr in #1391
• Update training_zoo.mdx by @yuvalkirstain in #1397

New Contributors

• @HeyangQin made their first contribution in #1251
• @tensimiku made their first contribution in #1259
• @xyfJASON made their first contribution in #1268
• @StandardAI made their first contribution in #1273
• @guspan-tanadi made their first contribution in #1272
• @anentropic made their first contribution in #1297
• @aashiqmuhamed made their first contribution in #1312
• @williamberman made their first contribution in #1315
• @xloem made their first contribution in #1319
• @flukeskywalker made their first contribution in #1304
• @Dango233 made their first contribution in #1348
• @apbard made their first contribution in #1356
• @kiyoon made their first contribution in #1363
• @sam-hieken made their first contribution in #1367
• @ys-eric-choi made their first contribution in #1386
• @tcapelle made their first contribution in #962
• @yuvalkirstain made their first contribution in #1397

Full Changelog: v0.18.0...v0.19.0