Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]Ark0.4.1 multi_gou_tutorial.py run error #207

Open
shenyanmei2020 opened this issue Jan 25, 2024 · 1 comment
Open

[Bug]Ark0.4.1 multi_gou_tutorial.py run error #207

shenyanmei2020 opened this issue Jan 25, 2024 · 1 comment

Comments

@shenyanmei2020
Copy link

Describe the bug
ark0.4.1: run multi_gou_tutorial.py fail in sched_default.cc
line393 in configure_gpu_buf, tensor.cc line246 in update_pads, errors as follow:
invalid padding detected. This is likely caused because one GPU buffer is used by multiple operators that require different padding. A possible workaround is to let each operator use a different buffer by creating a new tensor rather than overwriting an existing tensor op name:send.

To Reproduce
run multi_gou_tutorial.py in ark0.4.1

Expected behavior

  1. explain why has the error;
  2. what relationship "ldims, type_bytes, tile" between ref_tensor and this_tensor satisfy in updae_pads?

System (please complete the following information):

  • ark0.4.1
  • OS: [e.g. Ubuntu18.04]
  • GPU [A100]
  • Networking Environment [Single-node, Multi-gpu]

Additional context
Add any other context about the problem here.

@chhwang
Copy link
Contributor

chhwang commented Feb 26, 2024

@shenyanmei2020 Sorry for delayed response, and thank you for reporting. This is expected and we need to change the tutorial code. Let us response again with a fix soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants