-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How XGBoost sample_weights alternate the objective function formula? #10297
Comments
Yes, it's multiplied into gradients. It's also used in some other places like sketching. |
Great, thanks for the response! |
Feel free to reopen if there are further questions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From my understanding and reading post #144
when sample_weights are used, then first and second gradient
g_i and h_i are multiplied with sample_weights so that for each instance i:
grad_xgb[i] = grad[i] * case_weight[i]
hess_xgb[i] = hess[i] * case_weight[i]
The 2016 paper ‘XGBoost: A Scalable Tree Boosting System’ states that,
if we remove the constant terms, the objective function at step t is given in eq.3
The question is: How eq.3 is alternated when using sample weights?
Is it as simple as that:
(g_i*case_weight[i])*ft(xi)+1/2(h_i * case_weight[i]) (ft)power of 2(xi) + Ω(ft)
In other words, just substituting g_i, and h_i with grad[i] * case_weight[i] and hess[i] * case_weight[i] respectively?
Unfortunately, the links provided in post 144 are broken.
The text was updated successfully, but these errors were encountered: