Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: test result may be flaky #14691

Closed
wants to merge 1 commit into from

Conversation

Shane-XB-Qian
Copy link
Contributor

// fix: #14596

// fix: vim#14596

Signed-off-by: shane.xb.qian <[email protected]>
@Shane-XB-Qian
Copy link
Contributor Author

the prompt command line .... balabala... such thing after 'debug' running maybe very long, then it may show the shortmess msg, then such test term_getline(buf, 6) result may be flaky.

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

col=200 is wrong. The test must run within a terminal size of 24x80. What exact problem are you seeing?

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

The ticket description does not give any information. Tests must run on a 24x80 terminal size. So the max column size is 80, we cannot use more.

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

seems you were stubborn on this too much 😄 ,

Thank you!

here I remember it is the size of term in vim, not the vim itself.

If we only have a terminal size of 80x24, then there is no way to have a terminal window open within Vim which is 200 columns wide. How about you explain clearly what problem you are seeing and we look how to solve it differently?

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

ok, my last try: here 200 is the size of terminal in vim, not the terminal where the parent vim was running at. if not giving enough size the line would be wrapped.

Yes and I am not sure this is the correct solution. Because the terminal size (in which Vim runs) restricsts us what we can use as max terminal size within Vim.

So how about you explain clearly what problem you are seeing?

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@jamessan
Copy link
Contributor

jamessan commented May 1, 2024

Tests must run on a 24x80 terminal size. So the max column size is 80, we cannot use more.

No, we just require that the terminal is at least 24x80. It can be larger.

it cannot restricsts the size of terminal in vim

Vim can and does. :term ++cols=200 in a vim instance that isn't 200 columns wide will clamp down to whatever Vim's size is. You can verify that with :echo term_getsize(<bufnr>).

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

Tests must run on a 24x80 terminal size. So the max column size is 80, we cannot use more.

No, we just require that the terminal is at least 24x80. It can be larger.

Which means, it must also run in a 24x80 terminal.

it cannot restricsts the size of terminal in vim

Vim can and does. :term ++cols=200 in a vim instance that isn't 200 columns wide will clamp down to whatever Vim's size is. > You can verify that with :echo term_getsize().

I did. It returned 80 for a 24x80 terminal. So how does that fix the mentioned flakiness then?

@jamessan
Copy link
Contributor

jamessan commented May 1, 2024

So how does that fix the mentioned flakiness then?

No idea. :)

@chrisbra
Copy link
Member

chrisbra commented May 1, 2024

What I am trying to say is: Assuming that cols=200 fixes the flakiness when run in a terminal that is big enough, won't that cause still flakiness on a terminal that is small than 200 cols? If it still flaky in a smaller terminal, then I don't think this is the solution. If it however fixes the flakiness even in a smaller terminal window, why does it fix it?

I don't want to introduce a requirement, that the tests must be run in a terminal that is large enough for this particular test. I'd like to know what is the problem, if the term_getline() doesn't get the expected output because the lines wrap, then I think we should ensure that the terminal line isn't wrapped instead.

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@jamessan
Copy link
Contributor

jamessan commented May 1, 2024

it is 'a vim' in 'a terminal which it is in a vim' which 'that terminal' in 'a vim which in a parent terminal', a nested vim.

That doesn't matter. All of those nests will, by default, honor the size of their containing terminal and/or vim.

However, it is possible to make the :term in Vim use a larger size than Vim's size via 'termwinsize'.

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 1, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 2, 2024

Okay, I did run a test. I opened a 24x80 terminal.

vim --clean -c ':call term_start('bash', #{term_cols: 200})'
echo $COLUMNS returns 80
2->term_getsize() returns [10,80]

Doesn't seem to change anything 🤷‍♂️

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 2, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 2, 2024

Okay I checked it. It seems what is happening is this:

The test Run_Test_debug_running_out_of_lines() essentially runs an internal Vim instance using vim -S XdebugFunc (in a 6 lines tall terminal window) . When running this function, Vim will typically output:

Entering Debug mode.  Type "cont" to continue. 
command line..script /home/chrisbra/code/vim-upstream/src/testdir/XdebugFunc[15]..function <SNR>9_Crash
>

However if the path to the XdebugFunc script is too long, Vim will wrap the message (possibly a few times), which then may cause a "hit-enter" prompt, and so the WaitForAssert() call will timeout and fail.

We could pretend to be running inside of a 200 column window, by giving the cols: 200 option, which will then make Vim not wrap the lines. Or alternatively, we can increase the terminal window by a few more lines.

I am not sure, pretending a larger window won't cause issues later (because we had strange issues in the past when the vim window size was different from the terminal window size), so I would recommend to increase the terminal window by a few more lines. Using 10 lines should be enough for possible wrapping of the file name.

diff --git a/src/testdir/test_vim9_script.vim b/src/testdir/test_vim9_script.vim
index a3f21bed0..dc94ce1d0 100644
--- a/src/testdir/test_vim9_script.vim
+++ b/src/testdir/test_vim9_script.vim
@@ -4586,7 +4586,7 @@ def Run_Test_debug_running_out_of_lines()
       Crash()
   END
   writefile(lines, 'XdebugFunc', 'D')
-  var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 6, wait_for_ruler: 0})
+  var buf = g:RunVimInTerminal('-S XdebugFunc', {rows: 10, wait_for_ruler: 0})
   g:WaitForAssert(() => assert_match('^>', term_getline(buf, 6)))

   term_sendkeys(buf, "next\<CR>")

It would be great, if you could write something like this in the future, so it is clear what exact problem you are solving. Thanks.

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 2, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 2, 2024

ah correct, term_getline() needs to be adjusted as well.

  1. first of all, you donot know each time how many lines it would be wrapped.

correct, but perhaps 10 is enough for now, until someone reports another flakiness?

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 2, 2024 via email

@chrisbra
Copy link
Member

chrisbra commented May 2, 2024

so would that fix it or not?

@Shane-XB-Qian
Copy link
Contributor Author

Shane-XB-Qian commented May 3, 2024 via email

@chrisbra chrisbra closed this in 7edde3f May 4, 2024
@Shane-XB-Qian Shane-XB-Qian deleted the fix_flaky_test_res branch May 4, 2024 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

err: Test_debug_with_lambda & Test_debug_running_out_of_lines
3 participants