Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A possible mistake in document "GET TECHNICAL" on the broadcasting #1148

Open
tabe opened this issue Apr 12, 2024 · 0 comments
Open

A possible mistake in document "GET TECHNICAL" on the broadcasting #1148

tabe opened this issue Apr 12, 2024 · 0 comments

Comments

@tabe
Copy link

tabe commented Apr 12, 2024

I appreciate your "GET TECHNICAL" article in torch.mlverse.org for its clarity and efficiency to explain the training process of neural networks from scratch, which is very helpful.
Meanwhile, in its Tensor section the example of broadcasting reads:

We align array shapes, starting from the right.

Say we have two tensors, one of size 8x1x6x1, the other of size 7x1x5.

Here they are, right-aligned:

# t1, shape:     8  1  6  1
# t2, shape:        7  1  5

Starting to look from the right, the sizes along aligned axes either have to match exactly, or one of them has to be equal to 1: in which case the latter is broadcast to the larger one.

In the above example, this is the case for the second-from-last dimension. This now gives

# t1, shape:     8  1  6  1
# t2, shape:        7  6  5

But the broadcasting should start from the rightmost dimension, so

# t1, shape:     8  1  6  5
# t2, shape:        7  1  5

and then

# t1, shape:     8  1  6  5
# t2, shape:        7  6  5

In fact, in R

x <- torch_randn(c(8,1,6,1))
y <- torch_randn(c(7,1,5))
z <- x + y
z$size()

returns

[1] 8 7 6 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant