Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gatingset_to_flowjo gate not found; version problem? #126

Open
baj12 opened this issue Mar 7, 2021 · 24 comments
Open

gatingset_to_flowjo gate not found; version problem? #126

baj12 opened this issue Mar 7, 2021 · 24 comments

Comments

@baj12
Copy link

baj12 commented Mar 7, 2021

I want to export a gatingset to FlowJo. (the data origniates from a flowJo workspace, is a bit modified (added a gate and two markers)

I get the error message below, complaining about a population ("Treg_Memory-") that is not found.

> gatingset_to_flowjo(gs, outFile)
Using docker image rglab/gs-to-flowjo:devel to write FlowJo workspace...
Error in gatingset_to_flowjo(gs, outFile) : 
  terminate called after throwing an instance of 'std::domain_error'  what():  Treg_Memory- not found!
In addition: Warning message:
In gatingset_to_flowjo(gs, outFile) :
  docker image 'rglab/gs-to-flowjo:devel' is built with different cytolib version of from R package: 2.3.0 vs 2.3.7

I cannot find this population myself either:

> unique(gs_pop_get_count_fast(gs)$Population,boolean=T)
 [1] "/Lymphocytes"                                  "/Lymphocytes/SC"                              
 [3] "/Lymphocytes/SC/SC1"                           "/Lymphocytes/SC/SC1/CD4_P"                    
 [5] "/Lymphocytes/SC/SC1/CD4_P/CD4_P CD127_P"       "/Lymphocytes/SC/SC1/CD4_P/Treg"               
 [7] "/Lymphocytes/SC/SC1/CD4_P/Treg/Treg_Activated" "/Lymphocytes/SC/SC1/CD4_P/Treg/Treg_Memory"   
 [9] "/Lymphocytes/SC/SC1/CD4_P/Treg/Treg_Naive"     "/Lymphocytes/SC/SC1/CD8b_P"                   
[11] "/Lymphocytes/SC/SC1/SC1.other"                 "/Lymphocytes/SC/SC1/CD4_P/CD4_P.other"        
[13] "/Lymphocytes/SC/SC1/CD4_P/Treg/Treg.other"     "/Lymphocytes/SC/SC1/FSOMInput"  

I am also wondering if this has to do with the warning about 2.3.0 vs. 2.3.7.

Not sure what to test next. Any help appreciated.

Thanks,

Bernd

PS. not sure if these are related, but with the same data set, only a bit more modified, I get the following error message:

Error in gatingset_to_flowjo(gs2[20], outFile) : 
  terminate called after throwing an instance of 'std::runtime_error'  what():  Can't find any non-boolean children node under CD4_P CD127_P
@mikejiang
Copy link
Member

Can you share the minimum reproducible example?

@baj12
Copy link
Author

baj12 commented Mar 9, 2021

Can you try the file I sent you around the 18th?

gs = load_gs(sampleGS)
> gatingset_to_flowjo(gs, "test")

Using docker image rglab/gs-to-flowjo:devel to write FlowJo workspace...
Error in gatingset_to_flowjo(gs, "test") : 
  terminate called after throwing an instance of 'std::runtime_error'  what():  Can't find any non-boolean children node under CD4_P CD127_P
In addition: Warning message:
In gatingset_to_flowjo(gs, "test") :
  docker image 'rglab/gs-to-flowjo:devel' is built with different cytolib version of from R package: 2.3.0 vs 2.3.7
> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] flowWorkspace_4.3.7 CytoML_2.3.3       

loaded via a namespace (and not attached):
 [1] Biobase_2.50.0              httr_1.4.2                  tidyr_1.1.2.9000            ConsensusClusterPlus_1.54.0 jsonlite_1.7.2              RcppParallel_5.0.3         
 [7] assertthat_0.2.1            stats4_4.0.3                latticeExtra_0.6-29         RBGL_1.66.0                 yaml_2.2.1                  pillar_1.5.1               
[13] lattice_0.20-41             glue_1.4.2                  digest_0.6.27               RColorBrewer_1.1-2          colorspace_2.0-0            ggcyto_1.18.0              
[19] htmltools_0.5.1.1           plyr_1.8.6                  XML_3.99-0.5                pkgconfig_2.0.3             tsne_0.1-3                  calibrate_1.7.7            
[25] zlibbioc_1.36.0             purrr_0.3.4                 flowCore_2.3.2              scales_1.1.1                jpeg_0.1-8.1                pracma_2.3.3               
[31] tibble_3.1.0                aws.s3_0.3.21               generics_0.1.0              ggplot2_3.3.3               ellipsis_0.3.1              DT_0.17                    
[37] BiocGenerics_0.36.0         hexbin_1.28.2               magrittr_2.0.1              crayon_1.4.1                evaluate_0.14               fansi_0.4.2                
[43] MASS_7.3-53                 xml2_1.3.2                  graph_1.68.0                tools_4.0.3                 data.table_1.14.0           ncdfFlow_2.36.0            
[49] lifecycle_1.0.0             matrixStats_0.58.0          S4Vectors_0.28.1            munsell_0.5.0               cluster_2.1.0               compiler_4.0.3             
[55] rlang_0.4.10                debugme_1.1.0               grid_4.0.3                  htmlwidgets_1.5.3           aws.signature_0.6.0         igraph_1.2.6               
[61] base64enc_0.1-3             rmarkdown_2.6               cytolib_2.3.7               gtable_0.3.0                DBI_1.1.1                   curl_4.3                   
[67] R6_2.5.0                    RProtoBufLib_2.3.4          gridExtra_2.3               knitr_1.31                  dplyr_1.0.5                 utf8_1.1.4                 
[73] FlowSOM_1.22.0              Rgraphviz_2.34.0            stringi_1.5.3               parallel_4.0.3              Rcpp_1.0.6                  vctrs_0.3.6                
[79] png_0.1-7                   tidyselect_1.1.0            xfun_0.20                  
> 

@mikejiang
Copy link
Member

I think I know where the problem is. Let me tackle this and get back to you once we have a solution.

@mikejiang
Copy link
Member

I've pushed the fix. You will need to pull the latest CytoML from github master and pull the new docker image

docker pull public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest

@baj12
Copy link
Author

baj12 commented Mar 11, 2021

I have pulled the image but cannot run it:

> system2("docker", paste0("run ", docker.image, " --cytolib-version"), stdout = TRUE)
character(0)
attr(,"status")
[1] 132
attr(,"errmsg")
[1] "Resource temporarily unavailable"
Warning message:
In system2("docker", paste0("run ", docker.image, " --cytolib-version"),  :
  running command ''docker' run public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest --cytolib-version' had status 132

I am not a docker expert and don't know what to do with status 132 or "Resource temporarily unavailable"

@baj12
Copy link
Author

baj12 commented Mar 11, 2021

Originally I ran into this problem:

> gatingset_to_flowjo(fp, "test.wsp")
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error: '/var/folders/d_/g1jsd9v96mvc9n0ds29v7m8m0000gn/T//RtmpOtPBY0/filee6481c5c4ea0' does not exist.
In addition: Warning message:
In system2("docker", paste0("run ", docker_img, " --cytolib-version"),  :
  running command ''docker' run public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest --cytolib-version' had status 132

I am not sure why the temp file doesn't exist as well.

Thx for looking into this.

@mikejiang
Copy link
Member

I don't know how to reproduce your error, here is what you should get when you run the command at termainal

CytoML$ docker run public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest --cytolib-version
2.3.2

@baj12
Copy link
Author

baj12 commented Mar 12, 2021

Hi Mike,
I ran quite a few tests on quite a few machines. Bottom line, on some computers the docker file works on others not. Most importantly, it doesn't work on any of the machines I have access to.
Within the docker file I tracked down the problem to the following:

#0  0x0000563c3876ac45 in google::protobuf::internal::ReadSizeFallback(char const*, unsigned int) ()
(gdb) up
#1  0x0000563c385d7b55 in google::protobuf::internal::ReadSize(char const**) ()
(gdb) up
#2  0x0000563c386ccd7a in char const* google::protobuf::internal::ParseContext::ParseMessage<google::protobuf::DescriptorProto>(google::protobuf::DescriptorProto*, char const*) ()
(gdb) up
#3  0x0000563c3869abb4 in google::protobuf::FileDescriptorProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*) ()
(gdb) up
#4  0x0000563c38771e9b in bool google::protobuf::internal::MergeFromImpl<false>(google::protobuf::StringPiece, google::protobuf::MessageLite*, google::protobuf::MessageLite::ParseFlags)
    ()
(gdb) up
#5  0x0000563c38772c88 in bool google::protobuf::MessageLite::ParseFrom<(google::protobuf::MessageLite::ParseFlags)1, google::protobuf::StringPiece>(google::protobuf::StringPiece const&) ()
(gdb) up
#6  0x0000563c3877067d in google::protobuf::MessageLite::ParseFromArray(void const*, int) ()
(gdb) up
#7  0x0000563c386d7733 in google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) ()
(gdb) up
#8  0x0000563c3863a065 in google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) ()
(gdb) up
#9  0x0000563c385f79e4 in google::protobuf::(anonymous namespace)::AddDescriptorsImpl(google::protobuf::internal::DescriptorTable const*) ()
(gdb) up
#10 0x0000563c385f7a71 in google::protobuf::internal::AddDescriptors(google::protobuf::internal::DescriptorTable const*) ()
(gdb) up
#11 0x0000563c385d672e in __static_initialization_and_destruction_0(int, int) ()
(gdb) up
#12 0x0000563c385d674f in _GLOBAL__sub_I_GatingSet.pb.cc ()
(gdb) up
#13 0x0000563c3969844d in __libc_csu_init ()
(gdb) up
#14 0x00007fa0c9d7c040 in __libc_start_main (main=0x563c38402a81 <main>, argc=2, argv=0x7ffdf7ec71c8, init=0x563c39698400 <__libc_csu_init>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7ffdf7ec71b8) at ../csu/libc-start.c:264
264	../csu/libc-start.c: No such file or directory.
(gdb) up
#15 0x0000563c384025ee in _start ()

But, again, this doesn't mean much to me.
I have updated and upgraded everything but the OS. (I am still working on Mojave, 10.14.6 )

Before upgrading the OS, which I really like to push to some time with less stress, I would like to ask you for the script to create the docker file. Maybe I can get this working.

Thanks for you consideration.

@mikejiang
Copy link
Member

docker is os and machine independent , shouldn't behave differently. Here is a reproducible example see if it works for you

library(flowWorkspace)
library(CytoML)
dataDir <- system.file("extdata",package="flowWorkspaceData")
gs_dir <- list.files(dataDir, pattern = "gs_manual",full = TRUE)
gs <- load_gs(gs_dir)
outFile <- tempfile(fileext = ".wsp")
gatingset_to_flowjo(gs, outFile)

@baj12
Copy link
Author

baj12 commented Mar 12, 2021

library(flowWorkspace)
> library(CytoML)
> dataDir <- system.file("extdata",package="flowWorkspaceData")
> gs_dir <- list.files(dataDir, pattern = "gs_manual",full = TRUE)
> gs <- load_gs(gs_dir)
> outFile <- tempfile(fileext = ".wsp")
> gatingset_to_flowjo(gs, outFile)
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error: '/var/folders/d_/g1jsd9v96mvc9n0ds29v7m8m0000gn/T//RtmpFSgzEg/file268f7640fcc' does not exist.
In addition: Warning message:
In system2("docker", paste0("run ", docker_img, " --cytolib-version"),  :
  running command ''docker' run public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest --cytolib-version' had status 132

Unfortunately, there are some specific issues with MAC as it is running a different VM.
https://www.reddit.com/r/docker/comments/bltx02/container_exits_with_132_with_no_logs/
https://bugzilla.redhat.com/show_bug.cgi?id=1795574

These sites are not 100% matching, but they show some aspects of the issue. I can access the executable within the docker file, but it would be helpful to see the docker file to create the image such that I can verify the machine and then the executable. I guess there are certain libraries that are responsible for the interaction between the VM, mac OS and Linux which are not behaving.

Btw. I was able to verify that the mounting of files works, i.e. I can access the the files from my Mac from within the docker image. (at /gs)

@mikejiang
Copy link
Member

here is the dockerfile

#set up running environment for gs-to-wsp
FROM ubuntu:20.04
#prepare build env
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update
RUN apt-get install -y libblas-dev liblapack-dev libgomp1


COPY gs-to-wsp /

# Pre-create mount points for I/O
RUN mkdir /gs /out
ENTRYPOINT ["/gs-to-wsp"]
CMD ['--help']

I am not sure how much help you can get from it, since the gs-to-wsp executable is built and copied from the private host machine. It shouldn't really matter since I expect the docker image has all the environment it needs to run the executable.
One thing you can do is get inside of container and cp your gs to it, and run the gs-to-wsp in the container to see if it works

docker run -it --entrypoint=/bin/bash public.ecr.aws/x4k5d9i7/cytoverse/gs-to-ws
root@50202e6faf2c:/# ./gs-to-wsp --version

1 similar comment
@mikejiang
Copy link
Member

here is the dockerfile

#set up running environment for gs-to-wsp
FROM ubuntu:20.04
#prepare build env
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update
RUN apt-get install -y libblas-dev liblapack-dev libgomp1


COPY gs-to-wsp /

# Pre-create mount points for I/O
RUN mkdir /gs /out
ENTRYPOINT ["/gs-to-wsp"]
CMD ['--help']

I am not sure how much help you can get from it, since the gs-to-wsp executable is built and copied from the private host machine. It shouldn't really matter since I expect the docker image has all the environment it needs to run the executable.
One thing you can do is get inside of container and cp your gs to it, and run the gs-to-wsp in the container to see if it works

docker run -it --entrypoint=/bin/bash public.ecr.aws/x4k5d9i7/cytoverse/gs-to-ws
root@50202e6faf2c:/# ./gs-to-wsp --version

@baj12
Copy link
Author

baj12 commented Mar 14, 2021

Sorry, still not working.

As a last resort, I even updated the OS to Big Sur.

I installed gdb within the VM and traced back the illegal instruction to:

0x563c325d4c45 <_ZN6google8protobuf8internal16ReadSizeFallbackEPKcj+67> shlx %eax,%ecx,%edx

for the shlx command I found out that it might be that my computer is too old (2013, Ivy Bridge EP, Xeon E5) is not supporting AVX2.

(I also tried copying the executable into a new VM, without luck.)

Would it be possible to compile it on my side within the VM or even directly on MacOS?

@mikejiang
Copy link
Member

Does it mean that docker in general isn't working in your mac vm? Can you enter bash mode by docker run -it --entrypoint=/bin/bash public.ecr.aws/x4k5d9i7/cytoverse/gs-to-ws at all?

@baj12
Copy link
Author

baj12 commented Mar 15, 2021

docker is working. I can run R within Docker. Version 2.0 is running normally:

docker run rglab/gs-to-flowjo:2.0 --help
usage: gs-to-flowjo [--version] [--cytolib-version] [--help] <src> <dest> [--showHidden=no]


src			GatingSet archive directory
dest			output flowjo wsp file path
showHidden		whether to export the hidden populations

docker run rglab/gs-to-flowjo:2.0 --cytolib-version
2.0.2

Other docker images are running as well.
I would assume that you compiled the executable within the image on a machine with avx2 support. I don't know which switch would disable this during compilation. But since I checked this on several machines and only the older ones are broken I would conclude that the CPUs are not ready for this. Windows and Mac by the way.

@baj12
Copy link
Author

baj12 commented Mar 17, 2021

Mike,
I moved for this task to a newer CPU (from 2017) and docker works, but I still get the following error:

[1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library
> gatingset_to_flowjo(fp, "test.wsp")
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error in gatingset_to_flowjo(fp, "test.wsp") : 
  terminate called after throwing an instance of 'std::domain_error'  what():  Treg_Memory- not found!

the docker image produces the correct output:

 docker run public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest --cytolib-version
2.3.2

@baj12
Copy link
Author

baj12 commented Mar 22, 2021

Do you have any idea what could be the cause of the problem?
Since it is working with the sample data I might be able to change my data such that it works.
Thx

@baj12
Copy link
Author

baj12 commented Mar 28, 2021

Hi Mike,

not sure if you understood, that the VM is working and I found a more recent computer that runs the program.
But, I still get the same error message.
Can you reproduce this problem on your side?

thanks,

Bernd

@mikejiang
Copy link
Member

I've updated the docker image, with latest pull, your previous error should be gone. But this time, you should see

> library(CytoML)
> gs = load_gs("~/Downloads/sampleGS/")
> tmp = tempfile()
> gatingset_to_flowjo(gs, tmp, docker_img = 'public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp')
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp to write FlowJo workspace...
Error in gatingset_to_flowjo(gs, tmp, docker_img = "public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp") : 
  F0331 22:35:12.266316     1 flowjo_xml_node.cpp:507] grp.0.1: logical gate is not supported***

which tells you that your gs has the logical gate from grp.0.1 that is not supported by flowJo. Can you tell me how it is generated? Is it from flowjo or added by you with our tooling?

@baj12
Copy link
Author

baj12 commented Apr 1, 2021

Just as a side-note: yesterday I received a new computer, a MacBook pro.
Guess what happens when trying to install Docker:

Incompatible CPU detected
We are sorry, but your hardware is incompatible with Docker Desktop.
This version of Docker Desktop requires an Intel processor.

The chip is now called: "Apple M1"

There is only a RC (version 2) with quite a few unresolved issues.

This whole thing is turning into a nightmare.

I will try with a windows based computer from my son

@baj12
Copy link
Author

baj12 commented Apr 5, 2021

I hope we can get closer with this. It is not a "minimal" example but has all the elements I am using.

devtools::install_github("RGLab/CytoML")
install.packages("devtools")
library(devtools) #load it
devtools::install_github("RGLab/RProtoBufLib")

devtools::install_github("RGLab/cytolib")
install_github("RGLab/flowWorkspace")
devtools::install_github("RGLab/flowCore")
devtools::install_github("RGLab/flowWorkspaceData")
BiocManager::install("openCyto")
devtools::install_github("RGLab/openCyto")
devtools::install_github("RGLab/CytoML")

library(openCyto)
library(flowCore)
library(flowWorkspace)
library(CytoML)
fcsfiles <- list.files(pattern = "CytoTrol",
                       system.file("extdata",
                                   package = "flowWorkspaceData"),
                       full = TRUE)
(fs <- read.flowSet(fcsfiles))

gs <- GatingSet(fs)
gs

saveName = tempfile()
save_cytoset(gs, saveName)
cs = load_cytoset(saveName)
cs_unlock(cs)
for (sampleIdx in 1:length(gs)) {
  col2add = rep(0, gh_pop_get_count(gs[[sampleIdx]],"root"))
  #modify some col2add elements
  mm = matrix(ncol = 1, col2add, dimnames = list(NULL, "fsom.0.1"))
  mm[,1] = sample(1:10, replace = T, size = nrow(mm))
  cf_append_cols(cs[[sampleIdx]], mm)
}

gs_cyto_data(gs) <- cs
gsFile = "testGS"
save_gs(gs, path=gsFile, backend_opt = "copy")

gs_cleanup_temp(gs)
gs = load_gs(path=gsFile)
save_cytoset(gs, saveName)
cs = load_cytoset(saveName)
cs_unlock(cs)


metaGate <- function(fr, pp_res, channels = NA, metaNr, ...) {
  exprs(fr)[,"fsom.0.1"] %in%  metaNr
}
register_plugins(fun=metaGate, methodName="metaGate", type="gating")
parentGate = "root"
for (idx in 1:10) {
  gs_add_gating_method(gs, alias = paste0("grp.0.",idx), parent = parentGate,
                       gating_args = list(metaNr=c(idx)),
                       dims = "fsom.0.1", gating_method = "metaGate")
}

recompute(gs)
save_gs(gs = gs, path = saveName, backend_opt = "copy")
cs_cleanup_temp(cs)
gatingset_to_flowjo(gs, outFile = "outtest.jo")

with the following error message:

> gatingset_to_flowjo(gs, outFile = "outtest.jo")
Informationÿ: impossible de trouver des fichiers pour le(s) modŠle(s) sp‚cifi‚(s).
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error in gatingset_to_flowjo(gs, outFile = "outtest.jo") :
  terminate called after throwing an instance of 'std::domain_error'  what():  undefined getParam function!

@mikejiang
Copy link
Member

mikejiang commented Apr 6, 2021

First of all, if you pull the latest docker image , you should see

gatingset_to_flowjo(gs, outFile = "outtest.jo")
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error in gatingset_to_flowjo(gs, outFile = "outtest.jo") : 
  F0406 19:29:28.604566     1 flowjo_xml_node.cpp:507] grp.0.1: logical gate is not supported*

Secondly, you were creating the logical gate (which is simply a boolean vector) from opencyto, which is fine in itself within opencyto context. But flowJo doesn't have the equivalent gate type to represent these independent logical gate.

I think what you may want to do is to add these bool vector as a single factor , which will be treated the clustering results. And it can be handled by flowjo properly


g <- lapply(cs, function(fr) {
  as.factor(exprs(fr)[,"fsom.0.1"])# this will be split into multiple logical gates under the hood, but they still belong to the same clustering method
})
gs_pop_add(gs, g, name = "cluster", parent = "root")
gs_pop_get_children(gs, "root")
 [1] "/cluster_1"  "/cluster_2"  "/cluster_3"  "/cluster_4"  "/cluster_5"  "/cluster_6"  "/cluster_7"  "/cluster_8"  "/cluster_9" 
[10] "/cluster_10"
recompute(gs)
gatingset_to_flowjo(gs, outFile = "outtest.wsp")

 Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace

@audyavarac
Copy link

audyavarac commented Sep 14, 2022

I am getting the following error on M1 Pro chip Macbook Pro. I installed docker, got the latest dev version of flowCore, CytoML. But I am getting this weird msg on specifying image platform, for which i dont see any options to specify it. Please advise.

cyto_export(gsf, "ba48_1e5_flowjo_au.wsp" )
Saving GatingSet to flowJo workspace file...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Using docker image public.ecr.aws/x4k5d9i7/cytoverse/gs-to-wsp:latest to write FlowJo workspace...
Error in CytoML::gatingset_to_flowjo(x, save_as, ...) :
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

@dbrookeUAB
Copy link

I was able to fix the issue by adding --platform linux/amd64 to the system2 call in the original gatingset_to_flowjo function. The docker image was built for linux/amd64 and Apple Silicone uses linux/aarch64 .

❯ docker inspect -f '{{.Os}}/{{.Architecture}}' public.ecr.aws/ozette/cytoverse/gs-to-wsp
linux/amd64

❯ docker info -f '{{.OSType}}/{{.Architecture}}'
linux/aarch64 

As a temporary fix, copy and source the following

gatingset_to_flowjo <- function (gs, outFile, showHidden = FALSE, docker_img = NULL, ...) {
  
  res <- CytoML:::check_binary_status()
  if (res != "binary_ok") {
    res <- CytoML:::check_docker_status(docker_img)
  }
  if (!(res[1] %in% c("binary_ok", "docker_ok"))) 
    stop(res)
  if (is(gs, "GatingSet")) {
    tmp <- tempfile()
    suppressMessages(save_gs(gs, tmp, ...))
  }
  else tmp <- gs
  outFile <- gsub(" ", "\\\\ ", file.path(normalizePath(dirname(outFile)), 
                                          basename(outFile)))
  tmpfile <- tempfile()
  if (res[1] == "binary_ok") {
    message("Using local gs-to-flowjo binary to write FlowJo workspace...")
    res <- suppressWarnings(system2("gs-to-flowjo", paste0(" --src=", 
                                                           tmp, " --dest=", tmpfile, " --showHidden=", showHidden), 
                                    stderr = TRUE))
  }
  else {
    docker_img <- res[2]
    v1 <- packageVersion("cytolib")
    v2 <- system2("docker", paste0("run --platform linux/amd64", docker_img, " --cytolib-version"), 
                  stdout = TRUE)
    message(paste0("Using docker image ", docker_img, " to write FlowJo workspace..."))
    res <- suppressWarnings(system2("docker", paste0("run --platform linux/amd64", 
                                                     " -v ", tmp, ":/gs", " -v ", normalizePath(dirname(tmpfile)), 
                                                     ":/out ", docker_img, " --src=/gs --dest=/out/", 
                                                     basename(tmpfile), " --showHidden=", showHidden), 
                                    stderr = TRUE))
  }
  if (length(res) > 0) 
    stop(res)
  else {
    tree <- xml2::read_xml(tmpfile)
    CytoML:::add_version_info(tree)
    invisible(xml2::write_xml(tree, file = outFile))
  }
}

This will allow Docker to use emulation. If it still fails, make sure to install Rosetta 2.

Long term, I think it should be rebuilt for multiple-architectures. @mikejiang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants