customClientLoad.st fails to load on Windows

[HeSe]

I have a proposed chance to the customClientLoad.st script.

On my windows 7 with git bash the filetree paths seem incorrect

Instead of
filetree:///c/Users/
might have to be
filetree:///c:/Users/

Sebastian

[chisandrei]

What is the value of the variable GS_HOME? If you look at windowsGsDevKitInstallation.md you'll need to change GS_HOME to start with C:/

[HeSe]

Well,....
/c/Users/myuser/GsDevKit_home

That means that this is a filetree / git bash issue?

Sebastian

[HeSe]

If I set GS_HOME to:
C:/Users/ ...
I get the following error during isntallation:
MCRepositoryError: filetree:// repository 'C:\Users\myuser\GsDevKit_Home\shared\repos\gt4gemstone.smalltalk.ston\src' does not exist.

And If I set GS_HOME to:
/c:/Users ...
I get:
/c/Users/myuser/GsDevKit_home/bin/createClient: line 9: /C:/Users/myuser/GsDevKit_Home/bin/private/shFeedback: No such file or directory

[chisandrei]

What command do you use to load gt4gemstone into the stone? This line looks strange C:\Users\myuser\GsDevKit_Home\shared\repos\gt4gemstone.smalltalk.ston\src. Normally it should be C:\Users\myuser\GsDevKit_Home\shared\repos\gt4gemstone\.smalltalk.ston\src

[HeSe]

I use:
createClient -t pharo Gt4Gemstone -f -l -v Pharo5.0 -s gs_336 -z $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston

[HeSe]

One thing I realize is that with the createClient command as described before there is still only the 3.3.0 dlls (e.g. libgcirpc-3.3.0-32.dll) installed in GS_HOME/dev/clients/Gt4Gemstone

[HeSe]

Okay,... this is what I had to do to get a working image.
first command: createClient -t pharo Gt4Gemstone -f -l -v Pharo5.0 -s gs_336 -z $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston
This fails with an error, but I do have an image.
Then, I run: $GS_HOME/dev/clients/Gt4Gemstone/installClientGciLibraries 3.3.6
After command: startClient Gt4Gemstone -s Gt4Gemstone
In a Playground I manipulate and run the following script:

  | ci sessionName projectPath |
  [  
  'true' isEmpty
    ifFalse: [
      Metacello new
        baseline: 'Metacello';
        repository: 'filetree:///c:/Users/myuser/GsDevKit_home/shared/repos/metacello-work/repository';
        onConflict: [:ex | ex allow];
    load: #( 'batch' ).
      Metacello new
        baseline: 'Metacello';
        repository: 'filetree:///c:/Users/myuser/GsDevKit_home/shared/repos/metacello-work/repository';
    lock. "unconditionally lock Metacello" ].
    Metacello new
      baseline: 'Ston';
      repository: 'filetree:///c:/Users/myuser/GsDevKit_home/shared/repos/ston/repository';
  lock. "unconditionally lock Ston"
  'load' isEmpty
    ifTrue: [
      "If GsDevKitClient isn't present (GsDevKit_home not updated to latest), load only SmalltalkCI"
      Metacello new
        baseline: 'SmalltalkCI';
        repository: 'github://hpi-swa/smalltalkCI:master/repository';
        onConflict: [:ex | ex allow];
        load ]
    ifFalse: [
      "If GsDevKitClient is present, lock SmalltalkCI and load GsDevKitClient"
      Metacello new
        baseline: 'SmalltalkCI';
        repository: 'github://hpi-swa/smalltalkCI:master/repository';
        lock.  "explicitly lock before loading GsDevKitClient"
  Metacello new
        baseline: 'TodeClient';
        repository: 'filetree:///c:/Users/myuser/GsDevKit_home/shared/repos/todeClient';
        lock;
        onConflict: [:ex | ex allow];
        load: 'GsDevKitClient' ].
  'lock Metacello Repositories' isEmpty not
    ifTrue: [ (Smalltalk at: #GsDevKit_home) lockClientGsDevKitProjects ] ]
      on: Warning do: [:w | w resume ].
   projectPath := ''.
   [ ci := projectPath isEmpty
       ifTrue: [ (Smalltalk at: #SmalltalkCI) load: '/c:/Users/myuser/GsDevKit_home/shared/repos/gt4gemstone/.smalltalk.ston' projectDirectory: '/c:/Users/myuser/GsDevKit_home/shared/repos/gt4gemstone/']
   ifFalse: [ 
     (Smalltalk at: #SmalltalkCI) 
       load: '/c:/Users/myuser/GsDevKit_home/shared/repos/gt4gemstone/.smalltalk.ston'
       projectDirectory: projectPath ] ]
         on: MetacelloAllowLockedProjectChange
         do: [:ex | ex disallow "honor"].
  'gs_336' isEmpty
    ifTrue: [ 
      (ci compatibleConfigurationsFor: #gemstoneClient) do: [:configSpec |
        configSpec defaultSessionName ifNotNil: [:sessName |  sessionName := sessName ]]]
ifFalse: [ sessionName := 'gs_336' ].
  (Smalltalk at: #SCIGemStoneServerConfigSpec) defaultSessionName: sessionName

Changes are c: for filetree and "projectDirectory:" added otherwiese I would get an error that C:\Users\myuser\GsDevKit_Home\shared\repos\gt4gemstone.smalltalk.ston\src does not exist.

During load there is an error related to UnifiedFFI and a missing references, but you can resume.

After that I can define the session and connect.

Sebastian

[chisandrei]

Glad to hear that at least is works now :). Before executing createClient -t pharo Gt4Gemstone -f -l -v Pharo5.0 -s gs_336 -z $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston did you clone the gt4gemstone repo in $GS_HOME/shared/repos?

[HeSe]

Yes, I did the clone. But,  I actually tried both your master and dev_340 from dalehenrich

[chisandrei]

@dalehenrich do you have any idea what can go wrong here? Before following the instructions from the windows installation worked. Now it seems there is the following error:
MCRepositoryError: filetree:// repository 'C:\Users\myuser\GsDevKit_Home\shared\repos\gt4gemstone.smalltalk.ston\src' does not exist.
It is somehow strange that the path contains gt4gemstone.smalltalk.ston instead of gt4gemstone\smalltalk.ston. I did not try yet to also reproduce this on my machine.

[dalehenrich]

I will be off-line for 2-3 hours (dentist appointment and leaving shortly) ... I don't have a windows machine to test on, so I have fly blind when it comes to windows ... the only thing I can say is that I've recommended that folks use github/bash on windows and that might be the source or the problem, or there is a typo in a script somewhere, or ???

[HeSe]

Hi Dale!
This is a script that is generated during the Gt4Gemstone installation process. I do not know the scripts good enough, yet and can't give you a reliable information on that yet.
I suspect that during the script generation $GS_HOME is used to create the filetree entries in the load script.
Both of the following ways to set GS_HOME in git bash on windows result in the faulty load script.
export GS_HOME = pwd results in "/c/Users/..."
export GS_HOME = pwd -W results in "C:\Users..."
Both do result in a filetree:// entry in the load script that are not accepted.
The only entry that filetree can handle on windows and git bash seems to be:
filetree:///c:/Users/ ...

Sebastian

[dalehenrich]

@HeSe could you provide the command output from a failing createClient -t pharo Gt4Gemstone -f -l -v Pharo5.0 -s gs_336 -z $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston run? I provide quite a bit of debugging information in the output and without the log file I can olnly guess what is going and which script or scripts are involved ... I don't have a windows machine to test on, so I have to rely on the information from your failures to understand the issue.

[HeSe]

Here we go! The reason for the error is the filtree://c: thingy...

install.log

[dalehenrich]

@HeSe I received email with a different error condition ... am I supposed to completely ignore that error? It seems to indicate a script error that probably should be fixed ... unless???

[dalehenrich]

The fact that the todeLoad.st script appears to have completed it's build properly (you're deleted comment --- which actually has useful information ---) implies that the error is liekly not to be a filetree or metacello issue ... but an issue with something in the customClientLoad.st itself ...

[HeSe]

Not, yet!
The log represents the base error.
The error reported here is a result of me trying to run the customClientLoad.st script from within the created image after having changed the filetree entries to use "c:".

customClientLoad.st is a generated file. So that needs fixing. I do not know if the generation of that file is in your or Andrei's domain.

[dalehenrich]

... it looks like the Pharo stack has been truncated (a problem I have complained about in the past to no avail) so I cannot actually tell where the error is occurring ...

@chisandrei do you have any suggestions for @HeSe? to get better information from the pharo stack ... it's pretty useless to get a truncated stack when running batch jobs ...

I will read the script itself and see if I can find the offending line ... it is likely to be somewhere in that script...

[HeSe]

This is a copy from the shell

create Gt4Gemstone image using /c/Users/myuser/GsDevKit_home/shared/repos/gt 4gemstone/.smalltalk.ston
Error on or near line 18
'Errors in script loaded from C:\Users\myuser\GsDevKit_home\dev\clients\Gt4G emstone\customClientLoad.st'
SubscriptOutOfBounds: 2
ByteString(Object)>>errorSubscriptBounds:
ByteString(Object)>>at:
ByteString>>at:
ByteString(SequenceableCollection)>>second
WindowsStore>>printPath:on:
[ :out | self printPath: aPath on: out ] in WindowsStore(FileSystemStore)>>strin gFromPath: in Block: [ :out | self printPath: aPath on: out ]
String class(SequenceableCollection class)>>new:streamContents:
String class(SequenceableCollection class)>>streamContents:
WindowsStore(FileSystemStore)>>stringFromPath:
WindowsStore(DiskStore)>>basicEntryAt:
WindowsStore(DiskStore)>>isDirectory:
FileSystem>>isDirectory:
FileReference>>isDirectory
MCFileTreeFileSystemUtils class>>directoryExists:
MCFileTreeRepository>>repositoryProperties
MCFileTreeRepository>>directory:
[ :cl |
type = 'filetree'
ifTrue: [ | description headerSize |
description := aRepositorySpec description.
headerSize := 'filetree://' size.
^ cl new
directory:
(self
fileHandleOn:
(aRepositorySpec description
copyFrom: headerSize + 1
to: description size)) ] ] in MetacelloPharo30Platform(MetacelloPlatform)>>createRepository: in Block: [ :cl | ...
BlockClosure>>cull:
SystemDictionary(Dictionary)>>at:ifPresent:
SmalltalkImage>>at:ifPresent:
MetacelloPharo30Platform(MetacelloPlatform)>>createRepository:
MetacelloPharo30Platform(MetacelloPharoCommonPlatform)>>createRepository:
MetacelloMCBaselineProject(MetacelloMCProject)>>createRepository:
MetacelloRepositorySpec>>createRepository
[ aSpec createRepository ] in [ :aSpec |
| description repo |
description := aSpec description.
(repo := repositories
detect: [ :rep | rep description = description ]
ifNone: [ aSpec createRepository ]) ~~ nil
ifTrue: [ repos add: repo ] ] in MetacelloFetchingMCSpecLoader(Metacello CommonMCSpecLoader)>>repositoriesFrom:ignoreOverrides: in Block: [ aSpec createR epository ]
Array(Collection)>>detect:ifFound:ifNone:
Array(Collection)>>detect:ifNone:
[ :aSpec |
| description repo |
description := aSpec description.
(repo := repositories
detect: [ :rep | rep description = description ]
ifNone: [ aSpec createRepository ]) ~~ nil
ifTrue: [ repos add: repo ] ] in MetacelloFetchingMCSpecLoader(Metacello CommonMCSpecLoader)>>repositoriesFrom:ignoreOverrides: in Block: [ :aSpec | ...
Array(SequenceableCollection)>>do:
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>repositoriesFrom:ign oreOverrides:
'Errors in script loaded from C:\Users\myuser\GsDevKit_home\dev\clients\Gt4G emstone\customClientLoad.st'
SubscriptOutOfBounds: 2
ByteString(Object)>>errorSubscriptBounds:
ByteString(Object)>>at:
ByteString>>at:
ByteString(SequenceableCollection)>>second
WindowsStore>>printPath:on:
[ :out | self printPath: aPath on: out ] in WindowsStore(FileSystemStore)>>strin gFromPath: in Block: [ :out | self printPath: aPath on: out ]
String class(SequenceableCollection class)>>new:streamContents:
String class(SequenceableCollection class)>>streamContents:
WindowsStore(FileSystemStore)>>stringFromPath:
WindowsStore(DiskStore)>>basicEntryAt:
WindowsStore(DiskStore)>>isDirectory:
FileSystem>>isDirectory:
FileReference>>isDirectory
MCFileTreeFileSystemUtils class>>directoryExists:
MCFileTreeRepository>>repositoryProperties
MCFileTreeRepository>>directory:
[ :cl |
type = 'filetree'
ifTrue: [ | description headerSize |
description := aRepositorySpec description.
headerSize := 'filetree://' size.
^ cl new
directory:
(self
fileHandleOn:
(aRepositorySpec description
copyFrom: headerSize + 1
to: description size)) ] ] in MetacelloPharo30Platform(MetacelloPlatform)>>createRepository: in Block: [ :cl | ...
BlockClosure>>cull:
SystemDictionary(Dictionary)>>at:ifPresent:
SmalltalkImage>>at:ifPresent:
MetacelloPharo30Platform(MetacelloPlatform)>>createRepository:
MetacelloPharo30Platform(MetacelloPharoCommonPlatform)>>createRepository:
MetacelloMCBaselineProject(MetacelloMCProject)>>createRepository:
MetacelloRepositorySpec>>createRepository
[ aSpec createRepository ] in [ :aSpec |
| description repo |
description := aSpec description.
(repo := repositories
detect: [ :rep | rep description = description ]
ifNone: [ aSpec createRepository ]) ~~ nil
ifTrue: [ repos add: repo ] ] in MetacelloFetchingMCSpecLoader(Metacello CommonMCSpecLoader)>>repositoriesFrom:ignoreOverrides: in Block: [ aSpec createR epository ]
Array(Collection)>>detect:ifFound:ifNone:
Array(Collection)>>detect:ifNone:
[ :aSpec |
| description repo |
description := aSpec description.
(repo := repositories
detect: [ :rep | rep description = description ]
ifNone: [ aSpec createRepository ]) ~~ nil
ifTrue: [ repos add: repo ] ] in MetacelloFetchingMCSpecLoader(Metacello CommonMCSpecLoader)>>repositoriesFrom:ignoreOverrides: in Block: [ :aSpec | ...
Array(SequenceableCollection)>>do:
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>repositoriesFrom:ign oreOverrides:

.
./GemStone64BitClient3.3.6-x86.Windows_NT
./GemStone64BitClient3.3.6-x86.Windows_NT --> 3.3.6
cp: cannot stat '/c/Users/myuser/GsDevKit_home/shared/downloads/products/Gem Stone64BitClient3.3.6-x86.Windows_NT/bin32/msvcr*': No such file or directory
Error on or near line 97 :: installGci -d /c/Users/myuser/GsDevKit_home/dev/ clients/Gt4Gemstone -t pharo 3.3.6 :: installGci -d /c/Users/myuser/GsDevKit _home/dev/clients/Gt4Gemstone -t pharo 3.3.6
Error on or near line 57 :: scanProductsForInstallingGciLibs /c/Users/myuser /GsDevKit_home/dev/clients/Gt4Gemstone :: scanProductsForInstallingGciLibs /c/Us ers/myuser/GsDevKit_home/dev/clients/Gt4Gemstone
Error on or near line 272 :: createPharoTodeClient -c -l -z /c/Users/myuser/ GsDevKit_home/shared/repos/gt4gemstone/.smalltalk.ston -s gs_336 -v Pharo5.0 Gt4 Gemstone :: createPharoTodeClient -c -l -z /c/Users/myuser/GsDevKit_home/sha red/repos/gt4gemstone/.smalltalk.ston -s gs_336 -v Pharo5.0 Gt4Gemstone
Error on or near line 105 :: createClient -c -l -v Pharo5.0 -s gs_336 -z /c/User s/myuser/GsDevKit_home/shared/repos/gt4gemstone/.smalltalk.ston Gt4Gemstone :: createClient -c -l -v Pharo5.0 -s gs_336 -z /c/Users/myuser/GsDevKit_home /shared/repos/gt4gemstone/.smalltalk.ston Gt4Gemstone
Error on or near line 154 :: createClient -t pharo Gt4Gemstone -l -v Pharo5.0 -s gs_336 -z /c/Users/myuser/GsDevKit_home/shared/repos/gt4gemstone/.smalltalk .ston :: createClient -t pharo Gt4Gemstone -l -v Pharo5.0 -s gs_336 -z /c/Users/ myuser/GsDevKit_home/shared/repos/gt4gemstone/.smalltalk.ston

[dalehenrich]

@HeSe and @chisandrei am I right in guessing that the path that is causing the problem is the path passed in from the command line: createClient -t pharo Gt4Gemstone -f -l -v Pharo5.0 -s gs_336 -z $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston?

Since the pharo stack information from the log is useless ... I am guessing that script quoted by @HeSe in this comment and in that comment it seems that the error is related to the path being passed in on the command line: $GS_HOME/shared/repos/gt4gemstone/.smalltalk.ston. Is that right?

If so, then it just means that a Window's user needs to use a properly formed path for Pharo on Windows --- I have no idea what that would be ...

@HeSe you should be able to pass in a path on the command line that does not use $GS_HOME --- whatever path works for you ... before running into the /c/Users/myuser/GsDevKit_home/shared/downloads/products/GemStone64BitClient3.3.6-x86.Windows_NT/bin32/msvcr* error which now that I think about it also points back to a problem with $GS_HOME ... so it seems that perhaps setting $GS_HOME correctly for windows (I have not idea what that is ... for pharo) will make things work?

[dalehenrich]

@HeSe the pharo stack dump is truncated at the line: MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>repositoriesFrom:ign oreOverrides: ... for some reason pharo trucates stack reports from the debugger in the image as well as the stack dump files, so it is not possible to debug from a stack report when the stack gets to a certain size ...

[HeSe]

Hi Dale,

you are right and that is the point.

Both paths created on windows are not working with filetree:

export GS_HOME=pwd
results in
/c/Users/sheidbrink/GsDevKit_home/shared/repos

export GS_HOME=pwd -W
results in
C:/Users/sheidbrink/GsDevKit_home/shared/repos

Your tode1 installation seems to do some magic because here a ":" is added:

create tode1 image using /c/Users/myuser/GsDevKit_home/sys/default/pharo/win dows/todeLoad.st
Fetched -> BaselineOfMetacello-TravisCI.93 --- filetree://c:\Users\myuser\Gs DevKit_home\shared\repos\metacello-work\repository --- filetree://c:\Users\myuser\GsDevKit_home\shared\repos\metacello-work\repository
....

[dalehenrich]

My guess is that the $GS_HOME env var works within the bash scripts running with the github windows desktop code ... however, Pharo cannot use the $GS_HOME that works for Windows github ...

For tODE loads, I convert the paths using this code

[HeSe]

@chisandrei can you tell me in which code base the reference to customClientLoad.st can be found?
It is not in this repository...

[HeSe]

@dalehenrich thank you for your help!
Please check this issue it might not be 100% true that you convert properly for all tode loads...

[dalehenrich]

@HeSe the customClientLoad.st is generated by this script in the GsDevKit/GsDevKit_todeClient project

[HeSe]

okay, let me check if I can play with that

[dalehenrich]

@HeSe remember that I don't have a way to test any changes I make and I am curious why "only you" are having a problem ... All of these scripts apparently work for @chisandrei ... so now I want to know why '/c/Users/myuser/GsDevKit_home/shared/downloads/products/GemStone64BitClient3.3.6-x86.Windows_NT/bin32/msvcr*' creates problems for you and why @chisandrei doesn't have the problem building clients ...

[chisandrei]

@dalehenrich I was not having problems when using export GS_HOME=pwd -W to set the path in a git bash. However I was having similar issues when using `export GS_HOME=`pwd. Now I did not need to build a client from scratch on windows since some time, so there might be some other issues. I can try to reproduce this in the next days.

[HeSe]

@dalehenrich I just tried to alter the scripts so that /c/Users... would be come /c:/Users ... just as you discribed, but this will not work because there are other spots like "SmalltalkCI load:" that can't deal with /c/ , too.

I never tried to load GsDevKit_home or pharo from within git bash bsaed on MinGW64 on windoes before,...

That is why, I have no idea if that ever worked at all.

[dalehenrich]

I am not familiar at all with Windows, so I don't know what MinGW64 is, however, if you are not using GitHub for Windows then I am not surprised that you are having troubles, because that is the only Windows environment we "support".

[dalehenrich]

Here are the git install instructions for Windows ...

[HeSe]

Hi Dale, just to give you an update on this. These instructions do not work because of the missing git bash in the current github desktop package. I will try the old version you sent me right now.

[dalehenrich]

@HeSe sounds like a plan .. I will keep my fingers crossed:) ... without an acceptable bash shell implementation on windows, there will be quite a bit of work to port all of the scripts to windows ... my preference for the next major rework for GsDevKit_home would be to use guillep/Scale, but it isn't ported to windows yet ...