X must be non-null for matrix completion, but only when bootstrapping

[samzhang111]

Currently, if X is null for the matrix completion estimation, and bootstrapping is performed, then the code crashes with a cryptic Rcpp error:

Error in synth.boot(Y = Y, X = X, D = D, cl = cl, I = I, W = W, EM = EM, :
Bootstrap inference fails. Please check the data.

This happens, for example, on this line if cross-validation is not performed.

On the other hand, when I ran the MC estimation with se=FALSE, this crashing does not occur.

This may be the issue encountered in #65.

[xuyiqing]

Thank you. Does it work with data shipped with gsynth? Could you try inference = "nonparametric"?

…

On Mon, Sep 27, 2021 at 12:30 AM Sam Zhang ***@***.***> wrote: Currently, if X is null for the matrix completion estimation, and bootstrapping is performed, then the code crashes with a cryptic Rcpp error: Error in synth.boot(Y = Y, X = X, D = D, cl = cl, I = I, W = W, EM = EM, : Bootstrap inference fails. Please check the data. This happens, for example, on this line <https://github.com/xuyiqing/gsynth/blob/master/R/core.R#L1581> if cross-validation is not performed. On the other hand, when I ran the MC estimation with se=FALSE, this crashing does not occur. This may be the issue encountered in #65 <#65>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#70>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGDFFCMEQ63YXHR3SF3UEAMSPANCNFSM5EZ3Y7HQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[samzhang111]

Yes the issue reproduces with inference="nonparametric", and with the data from the package. Here is an exact line to replicate the issue:

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

fails, while

out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = FALSE, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

succeeds. (The difference from the example in the docs is that the formula is Y ~ D instead of Y ~ D + X1 + X2)

[xuyiqing]

Thank you. We'll look into this.

…

On Mon, Sep 27, 2021 at 1:02 AM Sam Zhang ***@***.***> wrote: Yes the issue reproduces with inference="nonparametric", and with the data from the package. Here is an exact line to replicate the issue: out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric") fails, while out <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = FALSE, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric") succeeds. (The difference from the example in the docs is that the formula is Y ~ D instead of Y ~ D + X1 + X2) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGE6YOKNHST2KVQ4E4DUEAQIPANCNFSM5EZ3Y7HQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[shishirdash]

Hello! I ran in to the same issue, curious if there has been any progress on it?

[xuyiqing]

Thank you. We'll look into this.

…

On Tue, Nov 9, 2021 at 4:42 PM Shishir Dash ***@***.***> wrote: Hello! I ran in to the same issue, curious if there has been any progress on it? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGBD56LQACD2HR4WXYDULG5WNANCNFSM5EZ3Y7HQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[PinchOfData]

I ran into the same issue, too.

[PinchOfData]

Any update on this issue?

In the meantime, a simple unorthodox fix is to add a random, uncorrelated variable to the estimation.

data(gsynth)
simdata$X3 <- runif(n=length(simdata$id))
out <- gsynth(Y ~ D + X3, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

[xuyiqing]

Sorry. Could you send a sample code/data to our email for us to replicate this error? Best, Yiqing

…

On Wed, Mar 23, 2022 at 6:01 AM Germain Gauthier ***@***.***> wrote: Any update on this issue? — Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGD4UNGWWRFNU7AQXWLVBMIZPANCNFSM5EZ3Y7HQ> . You are receiving this because you commented.Message ID: ***@***.***>

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[PinchOfData]

library(gsynth)
data(gsynth)

# Works with covariates
out1 <- gsynth(Y ~ D + X1 + X2, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

# Doesn't without
out2 <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric")

Will send it by email, too.

(great package btw)

[xuyiqing]

Thanks! Licheng, could you take a look?

…

On Wed, Mar 23, 2022 at 10:25 AM Germain Gauthier ***@***.***> wrote: library(gsynth) data(gsynth) # Works with covariates out1 <- gsynth(Y ~ D + X1 + X2, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric") # Doesn't without out2 <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 50, r = c(0, 5), CV = TRUE, force = "two-way", parallel = TRUE, cores = 4, inference = "nonparametric") Will send it by email, too. — Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGDLZSSMWDUTW4USLTDVBNHYRANCNFSM5EZ3Y7HQ> . You are receiving this because you commented.Message ID: ***@***.***>

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[mjpavlik]

Bumping this issue, as I am getting the same error. Thanks so much for this amazing package and for mc integration :)

[xuyiqing]

Sorry for the deley. We'll get to it soon!

…

On Wed, May 11, 2022 at 11:37 AM mjpavlik ***@***.***> wrote: Bumping this issue, as I am getting the same error. Thanks so much for this amazing package and for mc integration :) — Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGFFGMKRCF37TKPWRC3VJP46LANCNFSM5EZ3Y7HQ> . You are receiving this because you commented.Message ID: ***@***.***>

-- Yiqing Xu Assistant Professor Department of Political Science Stanford University https://yiqingxu.org/

[ChrisDoll]

Adding another workaround, reverting to version 1.1.4

# devtools
install.packages("devtools", version = "2.2.2", repos = 'http://cran.us.r-project.org', type = "source")


# Install for functioning commit: Aug 26, 2019, 1.1.4
devtools::install_github("xuyiqing/gsynth", ref = "461a761")

The error seems to be caused by:

               X.boot <- NULL
                if (p > 0) {
                    X.boot <- X[,boot.id,,drop = FALSE]
                }

in 'core.R'. When and why this is the case I don't know.

[MatthieuStigler]

I am having the same issue and can reproduce the error. It seems actually it comes from inter_fe_mc not handling a NULL X?

To check, insert a print(boot) at line 332 (just below if ("try-error" %in% class(boot)) {) of synth.boot, then:

library(gsynth)
data(gsynth)

fixInNamespace("synth.boot", "gsynth")
## introduce a print(boot) at line 332, just below if ("try-error" %in% class(boot)) {
out2 <- gsynth(Y ~ D, data = simdata, estimator = "mc", index = c("id","time"), se = TRUE, nboots = 1,  parallel = FALSE)

this will show error message:

Rcpp::not_compatible in inter_fe_mc(YY, Y0, X, II, beta0, hasF, lambda[1], force, tol): Not compatible with requested type: [type=NULL; target=double].>