在训练nanoGPT时出现了以下报错,请问我该怎么做 #17525
Unanswered
1os3
asked this question in
code help: NLP / ASR / TTS
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
(base) PS E:\GPT-2\nanoGPT> python train.py config/train_shakespeare_char.py --compile=False --block_size=64
Overriding config with config/train_shakespeare_char.py:
train a miniature character-level shakespeare model
good for debugging and playing on macbooks and such
out_dir = 'out-shakespeare-char'
eval_interval = 250 # keep frequent because we'll overfit
eval_iters = 200
log_interval = 10 # don't print too too often
we expect to overfit on this small dataset, so only save when val improves
always_save_checkpoint = False
wandb_log = False # override via command line if you like
wandb_project = 'shakespeare-char'
wandb_run_name = 'mini-gpt'
dataset = 'shakespeare_char'
gradient_accumulation_steps = 1
batch_size = 64
block_size = 256 # context of up to 256 previous characters
baby GPT model :)
n_layer = 6
n_head = 6
n_embd = 384
dropout = 0.2
learning_rate = 1e-3 # with baby networks can afford to go a bit higher
max_iters = 5000
lr_decay_iters = 5000 # make equal to max_iters usually
min_lr = 1e-4 # learning_rate / 10 usually
beta2 = 0.99 # make a bit bigger because number of tokens per iter is small
warmup_iters = 100 # not super necessary potentially
on macbook also add
device = 'cpu' # run on cpu only
compile = False # do not torch compile the model
Overriding: compile = False
Overriding: block_size = 64
tokens per iteration will be: 4,096
Traceback (most recent call last):
File "E:\GPT-2\nanoGPT\train.py", line 110, in
ctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype)
File "D:\PSAutoRecover\lib\site-packages\torch\amp\autocast_mode.py", line 234, in init
raise RuntimeError('Current CUDA Device does not support bfloat16. Please switch dtype to float16.')
RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.
最后一行好像是错误
Uploading nanoGPT…
Beta Was this translation helpful? Give feedback.
All reactions