-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ADAM bug when calculating the gradient in batches #178
Comments
So that would simply be to add |
I will do so👍 |
Co-authored-by: Peter Röseler <[email protected]> Co-authored-by: Steve Wood <[email protected]> Co-authored-by: Elena Peña Tapia <[email protected]>
Co-authored-by: Peter Röseler <[email protected]> Co-authored-by: Steve Wood <[email protected]> Co-authored-by: Elena Peña Tapia <[email protected]> (cherry picked from commit 6724b47)
#200) Co-authored-by: Peter Roeseler <[email protected]>
Environment
What is happening?
There is only one small error that prevents ADAM from calculating the gradient_num_diff for batches with max_evals_grouped. ADAM only gives "fun, self._eps" as argument when calling gradient_num_diff. This leads to max_evals_grouped=None, which leads to max_evals_grouped=1. Therefore, regardless of the call to set max_evals_grouped, max_evals_grouped=1 will always apply for ADAM.
Optimizer class method minimize:
ADAM class method gradient_num_diff:
Here are the corresponding files:
How can we reproduce the issue?
Create the ADAM optimizer. Call set_max_evals_grouped with any limit with the optimizer. Now call minimize with the optimizer and you will not notice any change in the runtime/CPU usage.
What should happen?
I could quickly fix the error by adding the missing argument.
Any suggestions?
No response
The text was updated successfully, but these errors were encountered: