New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: test result may be flaky #14691
Conversation
// fix: vim#14596 Signed-off-by: shane.xb.qian <[email protected]>
the prompt |
|
why 200 is wrong?
the default is 20*75, please check that runWithinTerm (a similar func name I am not nearing my PC) func detail.
the problem had been expressed at ticket description.
…--
shane.xb.qian
|
The ticket description does not give any information. Tests must run on a 24x80 terminal size. So the max column size is 80, we cannot use more. |
seems you were stubborn on this too much 😄 , here I remember it is the size of term in vim, not the vim itself.
…--
shane.xb.qian
|
Thank you!
If we only have a terminal size of 80x24, then there is no way to have a terminal window open within Vim which is 200 columns wide. How about you explain clearly what problem you are seeing and we look how to solve it differently? |
ok, my last try: here 200 is the size of terminal _in_ vim, not the terminal where the parent vim was running at. if not giving enough size the line would be wrapped.
…--
shane.xb.qian
|
Yes and I am not sure this is the correct solution. Because the terminal size (in which Vim runs) restricsts us what we can use as max terminal size within Vim. So how about you explain clearly what problem you are seeing? |
> ok, my last try: here 200 is the size of terminal _in_ vim, not the terminal where the parent vim was running at. if not giving enough size the line would be wrapped.
Because the terminal size (in which Vim runs) restricsts us what we can use as max terminal size within Vim.
this knowledge is wrong, it cannot `restricsts` the size of terminal _in_ vim, they seems can be different size.
// ok, this is all i can tell, and tried to fix or give the perhaps reason of such flaky test failure.
…--
shane.xb.qian
|
No, we just require that the terminal is at least 24x80. It can be larger.
Vim can and does. |
Which means, it must also run in a 24x80 terminal.
I did. It returned 80 for a 24x80 terminal. So how does that fix the mentioned flakiness then? |
No idea. :) |
What I am trying to say is: Assuming that I don't want to introduce a requirement, that the tests must be run in a terminal that is large enough for this particular test. I'd like to know what is the problem, if the |
> Tests must run on a 24x80 terminal size. So the max column size is 80, we cannot use more.
No, we just require that the terminal is [*at least* 24x80](min_term_size). It can be larger.
[min_term_size]: https://github.com/vim/vim/blob/2d919d2744a99c9bb9e79984e85b8e8f5ec14c07/src/testdir/runtest.vim#L62
> it cannot `restricsts` the size of terminal _in_ vim
Vim can and does.
yes.
`:term ++cols=200` in a vim instance that isn't 200 columns wide will clamp down to whatever Vim's size is. You can verify that with `:echo term_getsize(<bufnr>)`.
but case wrong, it is 'a vim' in 'a terminal which it is in a vim' which 'that terminal' in 'a vim which in a parent terminal', a nested vim.
call term_start('vim --clean', {'term_cols': 200})
…--
shane.xb.qian
|
What I am trying to say is: Assuming that `cols=200` fixes the flakiness when run in a terminal that is big enough, won't that cause still flakiness on a terminal that is small than 200 cols? If it still flaky in a smaller terminal, then I don't think this is the solution. If it however fixes the flakiness even in a smaller terminal window, why does it fix it?
I don't want to introduce a requirement, that the tests must be run in a terminal that is large enough for this particular test. I'd like to know what is the problem, if the `term_getline()` doesn't get the expected output because the lines wrap, then I think we should ensure that the terminal line isn't wrapped instead.
it is on you, i am just giving idea, and/or correct something.
…--
shane.xb.qian
|
That doesn't matter. All of those nests will, by default, honor the size of their containing terminal and/or vim. However, it is possible to make the |
no, `term_cols': 200` actually changed the nested COLUMNS
…--
shane.xb.qian
|
Okay, I did run a test. I opened a 24x80 terminal.
Doesn't seem to change anything 🤷♂️ |
it is nested _vim_, not the _bash_ -_-#
I said many times :-( , eventually we're testing the vim, not others.
…--
shane.xb.qian
|
Okay I checked it. It seems what is happening is this: The test
However if the path to the XdebugFunc script is too long, Vim will wrap the message (possibly a few times), which then may cause a "hit-enter" prompt, and so the WaitForAssert() call will timeout and fail. We could pretend to be running inside of a 200 column window, by giving the I am not sure, pretending a larger window won't cause issues later (because we had strange issues in the past when the vim window size was different from the terminal window size), so I would recommend to increase the terminal window by a few more lines. Using 10 lines should be enough for possible wrapping of the file name. diff --git a/src/testdir/test_vim9_script.vim b/src/testdir/test_vim9_script.vim
index a3f21bed0..dc94ce1d0 100644
--- a/src/testdir/test_vim9_script.vim
+++ b/src/testdir/test_vim9_script.vim
@@ -4586,7 +4586,7 @@ def Run_Test_debug_running_out_of_lines()
Crash()
END
writefile(lines, 'XdebugFunc', 'D')
- var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 6, wait_for_ruler: 0})
+ var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 10, wait_for_ruler: 0})
g:WaitForAssert(() => assert_match('^>', term_getline(buf, 6)))
term_sendkeys(buf, "next\<CR>") It would be great, if you could write something like this in the future, so it is clear what exact problem you are solving. Thanks. |
Okay I checked it. It seems what is happening is this:
The test `Run_Test_debug_running_out_of_lines()` essentially runs an internal Vim instance using `vim -S XdebugFunc` (in a 6 lines tall terminal window) . When running this function, Vim will typically output:
```
Entering Debug mode. Type "cont" to continue.
command line..script /home/chrisbra/code/vim-upstream/src/testdir/XdebugFunc[15]..function <SNR>9_Crash
```
However if the path to the XdebugFunc script is too long, Vim will wrap the message (possibly a few times), which then may cause a "hit-enter" prompt, and so the WaitForAssert() call will timeout and fail.
We could pretend to be running inside of a 200 column window, by giving the `cols: 200` option, which will then make Vim not wrap the lines.
this is the my PR here, tho 200 is a rough number.
Or alternatively, we can increase the terminal window by a few more lines.
I am not sure, pretending a larger window won't cause issues later (because we had strange issues in the past when the vim window size was different from the terminal window size),
yes, i knew 'geo' test in 'test_startup.vim' is a flaky test too, but that's on GUI (or gvim), it is testing using a assert_range(), but i think looks it was the menu size was fixed?
// anyway it's not to this CUI vim, though it sounds similar.
so I would recommend to increase the terminal window by a few more lines. Using 10 lines should be enough for possible wrapping of the file name.
```diff
diff --git a/src/testdir/test_vim9_script.vim b/src/testdir/test_vim9_script.vim
index a3f21bed0..dc94ce1d0 100644
--- a/src/testdir/test_vim9_script.vim
+++ b/src/testdir/test_vim9_script.vim
@@ -4586,7 +4586,7 @@ def Run_Test_debug_running_out_of_lines()
Crash()
END
writefile(lines, 'XdebugFunc', 'D')
- var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 6, wait_for_ruler: 0})
+ var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 10, wait_for_ruler: 0})
g:WaitForAssert(() => assert_match('^>', term_getline(buf, 6)))
term_sendkeys(buf, "next\<CR>")
```
i think this is not a right workaround:
1. first of all, you donot know each time how many lines it would be wrapped.
2. tho it extended to 10 lines, but it still get the 6'th line, so it still would not match.
…---
so unless you planed to fix/limit the nested vim issue, but that maybe make unknown consequence;
otherwise, my offer (this PR) perhaps was a better correction, or anyway this is an info about flaky res of these tests.
--
shane.xb.qian
|
ah correct,
correct, but perhaps 10 is enough for now, until someone reports another flakiness? |
ah correct, `term_getline()` needs to be adjusted as well.
> 1. first of all, you donot know each time how many lines it would be wrapped.
correct, but perhaps 10 is enough for now, until someone reports another flakiness?
actually i remembered i tried this before, but this looked the test would be more flaky if so:
1. for these here metioned test, the output actually was located at the bottom, 10 lines maybe helpless.
2. in our vim tests, there seems many test cases (including last time the gvim freeze case) were tested by this way:
a) window height to match the getline line number,
b) changing the height here perhaps not a good idea.
so unless you planed to fix/limit the nested vim issue, but that maybe make unknown consequence;
otherwise, my offer (this PR) perhaps was a better correction, or anyway this is an info about flaky res of these tests.
…--
shane.xb.qian
|
so would that fix it or not? |
so would that fix it or not?
yes, with adjusting the getline line number also, i have answered above;
but that's not my preferred way, have expressed above.
…--
shane.xb.qian
|
// fix: #14596