update fixed change for vl_ltr

[psky1111]

following the requirements and finish adjustments

[paddle-bot]

Thanks for your contribution!

[CLAassistant]

All committers have signed the CLA.

[TingquanGao]

还有一些问题，麻烦再看下

[TingquanGao]

这里是什么意思？

[psky1111]

已经在相关yaml文件中加入注释. 这里cache_root是在训练时保存的token缓存. 会自动生成"cached/"这个目录.

[TingquanGao]

感觉is_batch不太好，是不是可以换个变量名？

[psky1111]

已经更改变量名字

[psky1111]

好的,我马上看下

…

------------------ 原始邮件 ------------------ 发件人: "PaddlePaddle/PaddleClas" ***@***.***>; 发送时间: 2023年6月26日(星期一) 下午5:10 ***@***.***>; ***@***.******@***.***>; 主题: Re: [PaddlePaddle/PaddleClas] update fixed change for vl_ltr (PR #2828) @TingquanGao commented on this pull request. 还有一些问题，麻烦再看下 In ppcls/arch/backbone/model_zoo/vl_ltr.py: > @@ -0,0 +1,652 @@ +import os.path as osp +import os +from typing import Tuple, Union +#check +import paddle.distributed as dist 这个没有用到的话，就去掉吧。 In ppcls/arch/backbone/__init__.py: > @@ -88,6 +88,7 @@ from .model_zoo.adaface_ir_net import AdaFace_IR_18, AdaFace_IR_34, AdaFace_IR_50, AdaFace_IR_101, AdaFace_IR_152, AdaFace_IR_SE_50, AdaFace_IR_SE_101, AdaFace_IR_SE_152, AdaFace_IR_SE_200 from .model_zoo.wideresnet import WideResNet from .model_zoo.uniformer import UniFormer_small, UniFormer_small_plus, UniFormer_small_plus_dim64, UniFormer_base, UniFormer_base_ls +from .model_zoo.vl_ltr import CVLP_vit16,LGR_vit16 code style还是有问题，正常应该是“CVLP_vit16, LGR_vit16”，少了一个空格，建议再检查一下pre-commit In ppcls/arch/backbone/model_zoo/vl_ltr.py: > + new_size, + num_extra_tokens=num_extra_tokens) + pretrained_state_dict['pos_embed'] = new_pos_embed + + info = self.set_state_dict(pretrained_state_dict) + print('loaded pretrained clip.') + print(info) + + def build_attention_mask(self): + mask = paddle.empty((self.context_length, self.context_length)) + mask = paddle.full_like(mask, float("-inf")) + mask = paddle.triu(mask, diagonal=1) + return mask + + @Property + def dtype(self): 这个是在其他地方会用到吗？如果后续使用fp16训练的话，这样写可能会有问题。 In ppcls/arch/backbone/model_zoo/vl_ltr.py: > + for m in self.visual.parameters(): + if isinstance(m, nn.LayerNorm): + m.eval() + if isinstance(m, nn.BatchNorm2D): + m.eval() + if self.attn_grad is False: + print('freeze attn norm') + for m in self.text_block.attn.parameters(): + if isinstance(m, nn.LayerNorm): + m.eval() + if isinstance(m, nn.BatchNorm2D): + m.eval() + self.initialize_parameters() + + @Property + def dtype(self): 同上 In ppcls/configs/VL_LTR/VL_LTR_finetune.yaml: > + cache_root: "cached" + pretrained_clip: "vit" 这里是什么意思？ In ppcls/data/dataloader/imagenetLT_dataset.py: > +""" +CLIP text encoder and decoder +""" 这个注释是指的那里？ In ppcls/data/preprocess/batch_ops/batch_operators.py: > @@ -65,11 +65,12 @@ class MixupOperator(BatchOperator): """ - def __init__(self, class_num, alpha: float=1.): + def __init__(self, class_num, alpha: float=1., is_batch=False): 感觉is_batch不太好，是不是可以换个变量名？ On ppcls/loss/vltloss.py: 这里面的几个类都用到了吗？cross_entropy和label smooth相关的代码是否可以复用ppcls/loss/celoss.py中的？ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>