What do you envision from the terminal of the future?

[zadjii-msft]

We wanted to start a discussion to see what kinds of crazy ideas people might have for things that a terminal emulator might be capable of doing. Things that would make you more productive. Things that would enable command line applications to provide new experiences to users. Things that you might imagine every terminal does in 10 years, but no terminal is doing right now. What does your terminal environment look like in 2030?

I want folks to really go wild with their imaginations here. We've obviously got years of work planned here, and tons of great ideas we're already aware of and working on. But I'm looking for things that maybe no one else has thought of yet. The kinds of ideas that might take a few years for the Terminal to get to, but we might want to point ourselves in the direction of now.

For clarity, lets skip over things like the following. We've heard these requests before, they're good ideas, and they've likely got precedent. They're also the kinds of things we're already aware of and already planning (to the best of our ability).

• Better emoji support: Add emoji support to Windows Console #190 (et. al)
• Tab tear-off Add support for tab tearoff and tab merge #1256
• Continued perf improvements: [MEGATHREAD] AtlasEngine bugs and missing features (experimental.useAtlasEngine) #9999
• Sixel support: Feature Request: sixel graphics support #448
• KB text selection: Terminal: Support keyboard text selection & navigation #715
• Restore closed buffer contents: Restore previously closed session's state (Buffer contents) #961
• Broadcast input: Support broadcast input? #2634
• Automatic logging: megathread: Buffer exporting and logging #9700
• Automatic triggers: cmd: add environment variable to disable/enable 'Terminate batch job (Y/N)?' confirmation #217, Triggers(Including Text Strings) and Actions (internal or external calls)  #5916
• Dropdown customization: Allow dropdown menu customization in profiles.json #1571

Or really, anything on the first few pages of https://github.com/microsoft/terminal/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc, we're aware of.

Maybe you want collapsible regions in the terminal. Maybe a notebook-like commandline interface, where each command runs in its own buffer. I'm not necessarily looking for features that a shell (bash, pwsh) should provide, but I am interested in features that the terminal could provide for shells to add new features. I don't necessarily want Windows Terminal specific ideas, I want whatever you can imagine.

<discuss>

[zadjii-msft]

As an example of the kind of thing we're thinking about:

#3121 - shell-driven autocomplete menus that can provide rich info to the user, outside the limitations of the text grid itself. Sure, there are shells that implement similar features, but with the caveat that they need to print the menu to the buffer themselves. This leads to a lot of different implementations across different shells. Having the terminal display the suggestions also allows the terminal to provide semantic context to screen readers.

I really don't want this to be something that's Windows Terminal-specific, or based on heuristics. Shells have pretty great auto-completion that can already be configured. So I'd rather let the commandline app do the driving. Imagine if you could have vim/emacs/whoever provide the completeopt menu but in the terminal directly. Whenever {the next hot CLI editor} comes around, they could just leverage this autocomplete menu from the Terminal instead of having to roll all that rendering themselves. By making it shell-driven, it could be something any terminal emulator could also support.

[mubdi]

As an example of the kind of thing we're thinking about:

#3121 - shell-driven autocomplete menus that can provide rich info to the user, outside the limitations of the text grid itself. Sure, there are shells that implement similar features, but with the caveat that they need to print the menu to the buffer themselves. This leads to a lot of different implementations across different shells. Having the terminal display the suggestions also allows the terminal to provide semantic context to screen readers.

I really don't want this to be something that's Windows Terminal-specific, or based on heuristics. Shells have pretty great auto-completion that can already be configured. So I'd rather let the commandline app do the driving. Imagine if you could have vim/emacs/whoever provide the completeopt menu but in the terminal directly. Whenever {the next hot CLI editor} comes around, they could just leverage this autocomplete menu from the Terminal instead of having to roll all that rendering themselves. By making it shell-driven, it could be something any terminal emulator could also support.

Related to this, having some mechanism of discoverability for context-based documentation -- this would go a long way for helping on-board newer users to command-line based system. One could envision something like support for a Dash/Velocity-style documentation that would switch based on the context that the terminal is in (e.g., zsh/bash/ps, moving over to a different context for an ssh-session or screen, or even interactive environments like ipython). Something that would enable quick discoverability of what options one can/should use without having to switch back and forth to man pages.

[zadjii-msft]

@mubdi I actually really like that. Like, if the terminal knew what the commandline was being typed, it wouldn't be that hard to hook that up to manpages or Velocity
or whatever for that commandline. Those docs could totally slide in as a pane side-by-side with the terminal output.

[potatoqualitee]

Because you listed Better Emoji Support, I'd love centralized place to share and install themes, sorta like what VS Code has. Easy to distribute or download. That's one of my most used functions in VS Code. I have no idea if other terminals do theme stores -- Windows Terminal is the only one I use, other than default macOS terminal. Extensions in the store would also be fab.

[zadjii-msft]

BOY would @cinnamon-msft love to hear this. I think we had some discussion about a theme marketplace in the past. There's really great potential for some of this with the fragment extensions that we've already got, but there's some pretty mighty overhead between authoring a color scheme and publishing it in the Microsoft Store. There's gotta be an easier way.

[cinnamon-msft]

I really think just a domain we own would be easier than the Store. Could even do a GitHub website?

[zadjii-msft]

Oh for sure. It's JSON blobs. How hard could that be to manage?

[potatoqualitee]

if you go for the domain you own, perhaps you could use a GitHub Action to validate syntax then combine a directory of /themes into an all-themes.json then boop, put that on GH Pages with each merge to master 🥳

[miketheitguy]

Autocomplete and autocorrect.

[zadjii-msft]

Wouldn't these be things better suited to the shell, not the terminal? Like, PowerShell already has support for MenuComplete for a "graphical" autocomplete.

Maybe I'm missing something?

[DHowett]

Autocomplete is offered by some other terminals as well -- it's one step towards lifting the behavior up out of the shell. It's pretty compelling, thanks!

For example, this is what I see in Prompt for iOS:

[zadjii-msft]

dang now how are they doing that? There's bash-completions sure, but you think that's a "we actually know what the commandline you've typed is and are doing the completion ourselves"

I suppose theoretically you could modify readline to send the autocomplete sequence to the terminal

[DHowett]

If I had to guess I'd say they literally watch what you type, delimited by Enter, and tokenize it.

[miketheitguy]

Tab jumps through long commands. If I need to fix something in the middle of a 4 line command being able to jump quickly through it without holding an arrow for 20 seconds would be nice. :)

[zadjii-msft]

Depending on your shell, you can probably already do that. In cmd.exe and PowerShell, for example, ctrl+arrows will move the cursor a whole word at a time.

[OranguTech]

(Replying to a rather old comment I know, but to add detail:) Tab is (by default) used by a lot of shells (Powershell/Bash/etc) for command-completion.

[acgollapalli]

I think this might be a good place to start: Symbolics Demo: Kalman Reti

Well... If you really want to get crazy, that is. Objects can be returned as objects, which can be clicked on and altered. For instance, if I go ls . the thing returned is a directory list. But I can double click to go into the directory. So I guess, an object reference format which the terminal will render as a ui element, clickable and interactable via the mouse.

Language integration would be nice. Being able to move between say python objects and pass them around instead of bytestreams. But that might be more a shell thing.

Also, shells really shouldn't have to be purely text based. You should be able to go 'cat cat.jpg' and get a picture of a cat in a terminal. You should be able to have a terminal based application with a video embedded. You should be able to use your mouse on things that are supported, and move fluidly between the keyboard based terminal workflow and the graphical results.

[acgollapalli]

I don't think the value proposition of terminal based interfaces is in their usage of the keyboard per se, but in the simplicity of writing interfaces between programs and the terminal (as simple as command-line arguments or as complex as zork or nethack), and the simplicity of interacting with those interfaces. In a lot of ways, this proposal could add friction to that simplicity. I don't discount that concern. But I think that if your shell is something like xonsh or powershell (or maybe something based on babshka), then you're scripting using objects, not bytestreams. And these sorts of shells seem to work rather well. Objects have metadata, that tells you what it is, and because the shells have the sophistication to handle that, why shouldn't our terminals have the sophistication to make use of it? It just, setting aside the technical limitations for a minute it just doesn't make sense to me why I can't have the simple interface of typing in what I want the computer to do, and also of clicking on a thing when it's right there on the screen.

[zadjii-msft]

I suppose part of the trick then is how do we get these objects that have some internal representation to the terminal in a language-agnostic manner? For whatever reason, I can't think about this in any other way than "well duh, just serialize the objects to {something like JSON} and then write that to the terminal as a string of text" and now we haven't really made any progress.

Heck, powershell still converts the objects to text when you write-host.

<going way further afield>

Maybe we've gotta let there be a "fourth pipe" for transmitting richer data. Imagine (the horror of) writing a COM object to that fourth pipe, and then the Terminal on the other side can say "Okay, what are you? How would you like to be displayed?"

Of course, that object would have to stay alive in memory somewhere. Even ignoring something like ssh. When you're done executing ls, ls is gone - the process is totally lost. any objects it created would be gone too.

[acgollapalli]

Well, that's just it. ls may be gone, but it's output is not. That's the trick maybe.

The object conversion might need to be object rendering (to some kind of serializable representation the terminal understands), where the shell keeps the object alive while it's in visible history, but I think that's the shell's problem. The shell takes the objects, and sends terminal something like

$> ls

"""
^~{"render-value" : "<name-of-directory>",  
     "reference" : "<reference the shell understands>"
     "type" : "directory", 
     "full-path" : "<full-path>",
     "on-click" : "<stuff the terminal understands>", 
     "on-double-click" : "<shell> <command to rename directory>"}

^~{"render-value" : "<name-of-file>",  
     "type" : "jpg", 
     "reference" : "<reference the shell understands>",
     "full-path" : "<full-path>",
     "on-click" : "<terminal command to append <visual-depiction> to render-value>", 
     "on-double-click" : "<terminal command to display image in new-buffer>"}\n
"""

which displays as:

$> ls
<name-of-directory>
<name-of-file>

For an editable object, the shell would need to provide an on-edit.

To your point, I think that having an escape character(s) to tell the terminal "this is an object referring to a shell object" might be sufficient. Copying a thing from the terminal, might be copying the reference of the thing instead, which I guess would be an issue if your other programs don't understand that reference. But I think for most things, you don't really need to have the objects live, just some kind of reference for what it is on the filesystem or something that the shell can use to get a fuller representation of the thing, and, for that, simple, portable representations like paths will do. And if you do need a reference to a live object, you're probably copy-pasting right back into the terminal anyway, so it's a non-issue.

[naikrovek]

you folks seem to not know about "nushell" because it already does a lot of this, if not all of it.

https://www.nushell.sh/
https://github.com/nushell/nushell

[WSLUser]

Also, shells really shouldn't have to be purely text based. You should be able to go 'cat cat.jpg' and get a picture of a cat in a terminal.

There's been https://github.com/hackerb9/lsix. But WT needs to support Sixel first.

[cgranade]

I'd love to see kitty-style graphics support.

Examples in use:

• https://twitter.com/zkat__/status/1440866436436541452
• https://twitter.com/thingskatedid/status/1386086416665387008

[cgranade]

(Apologies for the redundancy, missed that this would likely fall under Sixel support.)

[zadjii-msft]

For sure. This is just another riff on "images in the terminal", let it be sixel(#448), the kitty protocol, or some other good protocol that hasn't been standardized yet.

[naikrovek]

after reading the "some other good protocol" link, i'm on board with abandoning support for sixel and supporting something like that. if it matters.

[docandrew]

How about “undo?” For bonus points, make something like “rm -rf /“ undoable using some kind of snapshot or diff/transaction/commit system.

[st0le]

A Customizable Status Bar

[zadjii-msft]

Already on the backlog 😉 #3459

[AshChak]

Came across this a few days back and thought it was cool.

HTTPS://Warp.dev

Maybe some features from this.👆🏼

[yostinso]

Password plugin support. (Make it Chrome-compatible so I can drop the 1Password extension in my terminal.) This almost works in Hyper.js but is janky.

Embeddable terminal. (A dream would be to let me tear off the tab, then drop it into VSCode and vice versa.)

Kind of like #12932 (reply in thread) describes for output hooks, a customizable translation layer for input. E.g. if I drop an image, paste as a bytestream instead of a path or convert my typed ASCII to e.g. hex.

vi cursor mode for select/copy/search

[MathiasMagnus]

How about copying from the terminal not as HTML formatted text, but as an SVG? Terminal UI libraries can implement the feature, but if the terminal itself did, it would be a lot more useful.

[naikrovek]

Can you please explain how copying as SVG would be useful? I don't understand the utility of that.

[zadjii-msft]

@MathiasMagnus Something like #1084 maybe?

[MathiasMagnus]

@zadjii-msft Something like 1084, yes. @naikrovek My main use case is copying stuff from the terminal to locations where I want to keep the original formatting but keep layout and not make it subject to word-wrapping and other formatting butchering but not turn it into raster. This comes up in two places:

• Making PowerPoint slides, but don't want to turn the presentation into a 10MB slidedeck, only because I got formatted console text on every slide.
• Copying very narrow but long parts of text (like one entire line) typically looks terrible in raster but is fine in vector. I'm sure you've seen thumbnails of PNGs on twitter where a 1600*40 pixel image is inserted.

In my opinion the ways one would want to copy console content is either:

• As plain text ✔
• As HTML5 ✔
• As raster ✔ (Snip tool or terminal capability)
• As vector ❌ (Absolutely no way to do it today)

All of these methods have pros/cons, and whenever I'm creating uni material, I often find looking for the last missing option. Having come across the tweet I linked is the exact kind of thing I'd like to embed into materials without losing quality or forcing DPI of my screen onto consumers.

Do let me know if you think it's nonsense or if I'm missing the forest for the trees.

[scintill]

I have a file called "terminal wishlist"! Here's some:

• Timestamped lines. Perhaps you can hover and see when it was printed. Easily seek between clusters of time.
• stdout/stderr coloring, maybe with even more streams than stdout/stderr -- again, maybe you can hover and see which stream a line came from.
• Filtering of contents (e.g. quick way to see only lines that match a regexp)
• Programs can output abbreviations -- mouseover to see expansion
• Quick and easy bookmarks (e.g. "I changed foo to use the frombulator here")
• Indicator of waiting on computation, I/O etc. It's embarrassing how often I've stared at a blinking cursor for tens of seconds, before realizing I was supposed to enter something before the command would finish. This is an app's UX problem addressable in other ways, but maybe this indicator would be useful too.

[zadjii-msft]

I love this stuff.

timestamped lines

this does seem like something the client app could do, but might be handy to let the terminal just do it for you. Looks like we've already got an issue for this: #9331.

stdout/stderr coloring

Maybe the hardest one. At least in windows, there's actually no internal difference between out and err. They both get hooked up to the same console backend, and they get hooked up without any indication to the console that they were out or err. That one's a thinker - how could we create multiple consoles, different ones for out and err, and interleave the output of the two streams in the correct order? Maybe that's not the right way to go about it.

quick way to see only lines that match a regexp

Oh interesting. Almost like the inverse of highlighting all the matches - instead, fade everything that doesn't match. I like it.

Programs can output abbreviations -- mouseover to see expansion

This one I don't think I understand - can you maybe give some more details? this seems like a really great idea, but I wanna make sure I'm imagining the same thing as you.

Quick and easy bookmarks

You're gonna love my current passion project #11000

Indicator of waiting on computation

You're gonna like #6700, #7955 et. al.

[scintill]

Thanks, I'm excited to see some of these under way already!

Almost like the inverse of highlighting all the matches - instead, fade everything that doesn't match.

Actually, I was thinking of completely hiding non-matching lines and closing the gaps. Maybe it would be in a separate window for clarity. Another way of explaining it, Unix-style, is /some/program | tee buffer.txt, the program ends, then later, grep someregexp buffer.txt -- but without having to manage buffer.txt manually and without having to have planned to do this beforehand. Maybe this fits in with the notebook-style ideas (though I'm not very familiar or comfortable with notebooks) -- you collected your output into a notebook cell, then later on do some more processing on that cell's contents, without necessarily altering the original cell contents. But it would be nice to do with the entire available scrollback buffer too.

Programs can output abbreviations -- mouseover to see expansion

Ok. The example that is coming to mind is fish shell's abbreviation of the working directory (at least in my configuration). The prompt might say ~/t/o/o/utils for the path /home/scintill/tmp/offlineimap-6.5.7/offlineimap/utils. I like the abbreviation, but it would also be handy to be able to hover over that (or other mode of interaction) and see what the full path was, especially if it was some time ago and you can't easily just pwd or such. This may not be a very interesting example, but the general feature could have lots of uses. Maybe it could be extended to blocks of text, such that an entire multi-line block could be abbreviated one way but be expandable/collapsible with more details available. The application would output both the abbreviation and the full text, annotated with control codes, and the terminal would handle expanding.

[lmlong]

highlight +1

[checkdigits]

Co-Pilot for terminal. Should work for batch file commands, PowerShell and Wsl distros. Can you imagine it being able to intelligently help you automate and complete all sorts of command line tasks?

Ten years into the future? It might not even take that long.

It would be epic.

[zadjii-msft]

@crutkas 👀

[naikrovek]

The ability to record input and output (inclusive of things like timing and unprintable and whitespace characters) into a file that can easily be replayed by other terminal users, without actually doing anything (just visually replaying what was recorded). Recording straight to MP4 doesn't always capture what I want. I want to be able to view the file and scroll through and see what happened, and I want to be able to replay that in real time inside some application so it looks like i'm watching a terminal, and I want to be able to share a video of it (if I have to make the video by screen-capturing a replay, that's fine; I don't need an MP4 renderer, really.) Capturing and recording straight to MP4 locks things like theme and font and sizing into the video and that isn't always desired.

[zadjii-msft]

@DHowett actually had this crazy prototype a while back that did exactly this! It would capture all the API calls to the console with the timing, so you could replay the exact set of API calls later. It was really cool, I wonder where that went.

idly: I wonder if the API calls would still work for different schemes - probably? I'm guessing that since the conpty's font and color table don't affect the Terminal's side, that it would work as you'd expect.

[DHowett]

Apart from the conhost dump/replay work (which happens on an API transaction level and is architecture/bitness-specific), I would love for us to get Asciinema support. They have a well-documented standard format for this, and a web-based player that can be themed and all.

[naikrovek]

Asciinema is fine, I just want a way to replay something the way it showed up on my screen, without necessarily recording it as a video.

Supporting Asciinema replay files would work, I'm sure, and I would PREFER not to need to visit a webpage to play the file, so adding support to Terminal for that would be lovely. Maybe not in scope, or is much less feasible that using the existing player.

Honestly, I'll take what I can get. 🙂

[PoleG97]

In materia of personalization, the option of "retro effect", wich are located under "default values > aparience", could be fine if we can define the value of the effect, as well as we do with opacity or blur effect.
The problem is that effect, in some personalizations of zsh (for example) get worse the apreciation of terminal. If we could choose the grade of effect would be great

Thanks

[naikrovek]

is there a sandbox we can use to author these shaders and see them in real time, like shadertoy?

[zadjii-msft]

Not really 😕 Some time ago I had a branch where the toggleShadeEffects action would "warm-reload" the shader, reparsing it each time shader effects got enabled. I don't think I ever got that shipped though.

[PoleG97]

I have a problem, could be that I have an error in the declaration because terminal sais me "Selected shader can't be found"

"experimental.retroTerminalEffect": true,
"experimental.pixelShaderPath": "Retro.h1s1",

This is a piece of JSON, but I'm not sure if it's fine from my part. I've test deleting the first line "experimental.retroTerminalEffect": true, an leaving only the second "experimental.pixelShaderPath": "Retro.h1s1", but still de same, so, apparently I'm doing it bad, how I should do?

And other thing, you said that I could choose the shader effect that I wants, where ir the list of available shaders?

Thanks a lot an congrats for your job

[naikrovek]

make sure the file extension of your shader and the file extension of the specified shader file name in the configuration match. (I don't think h1s1 is correct. it should be hlsl.)

Shaders are here.

[PoleG97]

Thats perfect, was exactly that (problem with my browser font), great, in additon I modified some params of .hlsl and know looks as I want.
Thanks for the help

[naikrovek]

would it be possible to support text "windowing" so that applications could define textual windows and their contents, but also keep things like Windows Narrator doing the right things, preventing it from reading the entire terminal left to right, line by line?

in the context of an application I am running from within Terminal, I imagine a windowing system where I define a window's top left coordinate, and a height and width, all of which are defined as values between 0 and 1. I specify a border, and a shadow, bg color, fg color, etc, except it's all rendered in UTF-8. scrolling within a window will scroll the contents of that window only, if the content is scrollable. the contents of each window are stored locally in RAM and not re-requested each time I scroll a single line in the terminal.

I am making no suggestions about any command-line applications, only full-terminal textual applications like text editors or IRC clients, things like that, including SSH applications.

I ♥️ TUIs and I hate the "dumb" nature of terminals which makes TUIs more difficult than they should be.

[naikrovek]

oh, and make it an open spec. ncurses and its ilk need to die, already.

[WSLUser]

You mean something like CSI 14 t or TIOCGWINSZ ioctl? There's a couple issues open requesting those. While ncurses itself is pretty old, there are some modern alternatives. In particular while ncurses is used for everything not directly supported, notcurses is a pretty good TUI library and can also be used for non-TUI's too.

[naikrovek]

I want a terminal control set extension or mode where I can:

1. define a thing and provide its contents
2. display the contents of that thing in a rectangle with a given location and size
3. use my mouse to hover over that rectangle and scroll the content without fetching new data from the remote computer/application

the nature of things like ... the entire history of terminals, which ncurses relies upon, preclude a solution like this.

I'm saying that I want something smarter than a dumb terminal, backwards compatibility be damned once you're in this new mode.

I want a text-only, bandwidth friendly way to use content-dense application user interfaces in the terminal. I don't want to fetch the entire screen every time anything scrolls into view.

[soloturn]

terminal of the future should be very good

• to help completing the command one types or speaks, using history of commands and other sources, displaying help and examples if no history is there.
• to help searching the output for errors
  it is a command line interface at the end - so better keep it simple :)

[rramsey]

If this topic isn't dead, I'd like to add drag and drop sftp support. You can already drag a folder to the command line to paste in a path, but if I'm in an sftp session, it would be useful to drag a folder into the terminal to upload it to the current remote directory.

Someone mentioned nushell, but you should also look at clink and oh my posh! for ideas.

[naikrovek]

I'd like to know what new development efforts/planned features came of this thread, if that isn't being kept secret.

[zadjii-msft]

Huh, not sure how much of this I can really comment on currently. This thread was intended as a "10 year" view, and that's what we got. That being said, we really only plan out a few months at a time - priorities seem to shift faster than that, so planning any farther than that is usually an exercise in futility.

There's basically a few camps that these responses were grouped into:

• Images in the Terminal
• New Terminal features
• Richer CLI application output

The first is pretty well understood problem, one that will just take a while to implement. I don't have much to add here.

The second is also fairly well understood, IMO, and the easiest path to delivering value to users. There we can add a lot of value quickly, without reinventing the entire CLI experience, or relying on third parties to implement new things we're providing support for. Broadly, the things that are higher on the backlog tend to fit into this category, and these are the ones that I believe my PMs would prefer I keep under wraps. I'll probably have some specs out early this year to help articulate some of these thoughts.

The last category was the one that was the least understood and the most open ended (and seemed like the threads you were most interested in 😉). It was also perhaps the most revolutionary. There was a lot of different ideas here on different ways for CLIs to provide richer TUIs. Some of that was things the console subsystem would need to support, while other seemed more like things that should be provided by a modern TUI library (like Terminal.Gui or Textualize). The closest concrete thing to come of that is some of the semantic markup stuff that Carlos is working on for a11y's sake. That's more of a start in the direction of defining richer regions of the output. IMO that would be best paired with a successor to ncurses, but that's not something we here on the Terminal team are gonna end up writing ourselves. This is the category of stuff that would really require a broad effort to coordinate with 3p devs and figure out exactly what needs to be built, rather than just doing it Field of Dreams style.

This is a real non-answer and I'm aware of that. A lot of this was already on the backlog in one shape or another, so this mostly just helped bump some things around. We've got a lot of other fundamentals to get out of the way before getting to some of these. I'd say stuff like that is really our priorities right now - tear out, better unicode, improved a11y support, stuff like that.

[naikrovek]

well, I wasn't expecting an answer at all, so thank you for what you did provide, even if it is vague and non-committal.

[debajyoti1990]

I think I can come with 2 crazy ideas :

1. Default inclusion of terminal icons.
2. An embedded terminal file manager similar to lf.

[necrosisy]

Add support for full font fallback sequences, so that text in other languages ​​can be displayed better.

[clinton-r]

Thank you for inviting us to post wish lists :D Here is mine. Sorry if any of it already exists or is on the TODO list.

Below, "I" am an application program written in C that uses a terminal emulator to interact with a user.

I would like all terminal emulators on all platforms to support everything below, in the same way.

(1.) I would like to be able to interact with the terminal only through an input stream of bytes and an output stream of bytes.
If I am running on the same computer as the terminal, these streams should be the stdin and stdout. So, I would like to be able to query and control everything about the terminal through vt control sequences. For example, there could be vt sequences to do:

• everything that GetConsoleMode()/SetConsoleMode()/tcgetattr()/tcsetattr() can do
• get/set the code page (note xterm has ESC % @ and ESC % G for this)
• get the screen size (rows/columns)
• etc.

Since Windows terminals can be in non-vt modes, it would be nice if there was a vt sequence that is recognized in all modes that would put the terminal into vt mode.

(2.) I would like a simple way to ask the terminal what common vt escape sequences it supports for display, using a vt sequence query/response.
I'd prefer it if the response just identifies one of a few standard sets of sequences, so I don't have to ask the terminal whether it supports each individual sequence that I'd like to use.
(Does the existing Device Attributes query/response already support this well enough?)
Perhaps the terminal could indicate its level of support of a standard set of sequences, say with a number. Just for example:
LEVEL 0
-LF, CR
LEVEL 1
-(LEVEL 0 stuff) + CUU, CUD, CUF, CUB, CUP, SU, SD, ED, EL, (and things of similar complexity)
LEVEL 2
-(LEVEL 1 stuff) + SGR bold/underline/negative + alternate screen buffer available + scrolling margins (...)
LEVEL 3
-(LEVEL 2 stuff) + SGR 8-colour foreground and background colours (...)
LEVEL 4
-(LEVEL 3 stuff) + SGR 256-colour...
etc.

(3.) I would like a simple way to ask the terminal exactly what it sends (to my stdin) for all key presses and mouse events.
I'd prefer it if the response just identifies one of a few standard sets of sequences, so I don't have to ask the terminal what it sends for each key.
There should never be more than one set of keys that generate the same sequence.

(4.) I would like a simple way to distinguish a press of the escape key from the start of an escape sequence.

(5.) Suggested alternative solution for 1 - 4:
Consider defining a reasonably useful and already broadly supported subset of vt sequences. Call it the One True Terminal. Add vt sequences to ask whether One True Terminal mode is supported and to switch to it.
OTT mode should do the things requested in 1. and 4. above.
(Suggest also: entering OTT mode switches the terminal to using utf-8.)
When in OTT mode, all terminals should behave identically when sent OTT vt sequences, and all send the same sequences to represent key presses and mouse events.
When I start running, the first thing I will send to stdout is the sequence to ask if OTT mode is supported, so if the terminal is not in VT mode it should switch to VT mode and respond.

(6.) I would like a way to ask the terminal to not redraw its screen until I finish telling it all the changes to make.

(7.) I would like a way to specify left and right bounds on a scrolling region, in addition to the top and bottom. I would also like to be able to erase that region in a simple way. I would also like to be able to command a scroll of that region left or right as well as up or down.

(8.) I would like a way to have the terminal append text to the normal screen buffer while the alternate screen buffer is being used and displayed.

(9.) I would like to be able to ask the terminal which key combinations are being intercepted by the terminal and OS so that I don't use those myself. (Even better would be if I can negotiate for the right to handle them myself.)

(10.) Could I also do file system and serial port I/O etc. on the terminal's computer through vt sequences? That way I could wait for input from any of them with a (blocking) call to e.g. fscanf().

Why?

(1.) Because in some cases those 2 streams might be all I have; e.g. if I'm running remotely through something like TELNET, or on a simple embedded system connected through RS232 serial. (BTW can Windows Terminal be used as a serial terminal?)

(2./3.) Because in some cases I won't be running on the same computer as the terminal, and might not have a way to check an environment variable on the terminal computer. And anyway sometimes $TERM is a lie. And I might not have access to ncurses/Termcap/Terminfo/whatever, or just prefer not to use them.

(1./2./3.) So I don't have to be compiled differently for every platform.

(6.) So my user will see all the changes happen instantly, and not see any of the intermediate states. Also, so that not as much time is spent updating the screen.

[j4james]

1. I think some of the things you're expecting here are the responsibility of the operating system, so there's not much the terminal can do about that. But you can already do things like setting the code page and querying the screen size with standard VT sequences.

2. A lot of this is covered by the DA1 report, which gives you the conformance level that the terminal supports, along with any extensions that is can handle beyond that base level. Supported modes can be queried with DECRQM, supported renditions with DECRQSS, and supported character sets with DECCIR. These queries may not be as simple as you would like, but they are at least part of an existing standard that has been around for decades.

3. This will depend on what modes you've selected, so I guess you can query the active modes to determine what keyboard and mouse reports you should be receiving, at least to some extent.

4. This could theoretically be achieved with S8C1T, but I don't know of anyone that supports it correctly for UTF-8 input. It's on my TODO list to implement in Windows Terminal, but there's not much point if nobody else supports it.

5. From past experience, I'd wager there is no chance of something like this being agreed upon. But it's probably safe to assume that most terminals support a subset of the VT100 sequences.

6. This is possible with synchronized output mode ?2026. It's not an official standard, but there are a fair number of terminals that support it, although not yet Windows Terminal.

7. This can be achieved with DECLRMM (left right margin mode) and DECSRLM (set left right margins).

8. If I've understood you correctly, this sort of thing can be done with VT pages, which are similar to the alternate screen buffer but much more powerful. You can choose to write to one page while another page is focused, and you can also copy content between pages. This is not yet supported in Windows Terminal, but it's on my TODO list.

9. I think there are VT sequences for something similar to this, but I don't know of any modern terminals that support them. Allowing an app to override system keys might also be considered a security risk.

10. I'm not sure I understand what you're suggesting here, but I think this is possibly outside the scope of the terminal.

[wcydo]

[DHowett]

@wcydo I deleted your previous comment because it was excessively rude, and I will be doing the same to this comment. Please refer to our code of conduct before you continue to engage in discussions here.

[wcydo]

Of course you did. It simply is mind boggling that you're engaging in all this while you're leaving behind the most important factor and clearly the most obvious drawbacks of using this thing on daily basis. It is clearly slow and under-performing. I toned down my response by writing another comment simply listing my "wild" ideas for the terminal by stating that it must perform faster. It must open up instantly. It must not take 5 seconds to decide if an incorrect command is given. It must not take 3 seconds to fetch user path. I believe these are fair and true major drawbacks with this project.

[DHowett]

You're yelling at the wrong people here. The terminal emulator doesn't control how long it takes to display your prompt or figure out what path you're running from, nor does it control how long it takes to reply telling you that a command wasn't found.

WT starts up in ~250ms from launch to interactivity, which is as far as we can tell near the "speed of light" for the frameworks we're using. If that is insufficient for you, you are welcome to use any other terminal emulator and go there to yell at them about how slowly your chosen shell starts up or processes your input.

To be entirely fair, I agree with some of the points you're making. Sure, PowerShell isn't the fastest shell on earth. That bothers me pretty much every day! The team who owns it and maintains their GitHub repository has also heard this feedback before and they certainly do care about it. After all, they live in PowerShell just as much as I do. They're impacted by it just as much as we are.

Practically speaking, this thread is not the appropriate place to vent your concerns. We--and our community--are allowed to discuss what cool things terminals could do in the years beyond 2023. It's not helpful and it's not on topic for you to show up in this specific thread and yell at us about things that the applications of today aren't doing up to your standards.

[zadjii-msft]

May I further refer you to What's the difference between a shell and a terminal?.

I hope that helps clarify!

[willeccles]

I would love to see a way to disable certain types of text rendering. For example, in my particular use-case, I would like to be able to disable italics terminal-wide. They look terrible with my font and having to keep figuring out how to disable them in every program is exhausting.

[markm77]

Anything to improve separation/visibility of input and output would be great.

I thought about a two-pane approach with input/history rows on left and output pane on right but probably this isn't the answer.

However maybe you could take some inspiration from IDEs:

• foldable output regions (including "collapse all")?
• sticky scroll to keep previous input command at top of pane when scrolling (see e.g. https://learn.microsoft.com/en-us/visualstudio/ide/editor-sticky-scroll?view=vs-2022)?

[tkmr-akhs]

I've been exploring the capabilities of Windows Terminal and am impressed with its versatility and the sendInput command. However, I've noticed a limitation that I believe could significantly enhance its functionality. I'm not sure if this has been proposed before, so I apologize if this is a duplicate request.

In Linux environments, we have the expect command, which allows for waiting for a specific output and responding to it accordingly. Implementing a similar feature in Windows Terminal would be incredibly beneficial. Here's a rough idea of how it could look in the settings:

Copy code
{
  "name": "Example",
  "command":
  {
    "action": "multipleActions",
    "actions": [
      { "action": "waitOutput", "output": "foo", "action": ... },
      { "action": "waitOutput", "output": "bar", "action": ... }
    ]
  },
  "keys": ""
}

This concept would allow for more dynamic and responsive scripting within the Terminal, adapting to varying outputs and scenarios. It would be a valuable addition, especially for those who require complex interaction with their terminal sessions.

I understand this might be outside the current scope of Windows Terminal, or perhaps there are complexities I'm not aware of. Any feedback or thoughts on this possibility would be greatly appreciated.

[zadjii-msft]

Something like #5916 maybe?

[tkmr-akhs]

I just read #5916. Indeed, it seems similar to the feature request you mentioned. In fact, the feature proposed in #5916 appears to be more abstract and flexible than my request.

While I can't fully imagine how the feature in #5916 would be implemented, it would presumably be more convenient if it's possible to toggle triggers on and off at will. For example, enabling them only when needed during work via the command palette, or activating them when opening a new tab with a specific profile. Additionally, it would be beneficial if the GUI could visually indicate whether a trigger is currently active.

Furthermore, it would be greatly appreciated if there was an option for each trigger to either continuously respond until manually disabled, or to become inactive after completing a series of responses.

Here's an example of what the JSON configuration might look like:

             "experimental.triggers": [
                 {
                     "type": "matchRegex",
                     "match": "Terminate batch job \\(Y/N\\)",
                     "command": { "action": "sendInput", "input": "y\n" },
+                    "persistent": true
                 }
             ]

[determin1st]

as a user of the terminal i expect terminal to report its capabilities in direct way, those are generally described by "modes", oldish DECRQM (DEC request mode) or <CSI>?<mode_numbers_here>$p escape sequence

for example when i launch "hyper" terminal, it identifies as "conhost", but i know that conhost supports mouse tracking very well, but that "hyper" itself dont let events to squeeze to me to parse them.

i have a probeMouse function that sends DECRQM for 1003,1002,1001,1000,9 modes, so i mark those as limited mouse tracking when terminal responds and one of them not zero and not permanently disabled. nix terminals (xterm, gnome-terminal) tell me that but no win terminal responds, so this probe is effectively useless step under winOs. i would like it be useful for better capability detection

[determin1st]

i said stupid thing that conhost supports mouse tracking, seem it doesnt. i was testing ConEmu silently injected intance.. still DECRQM will be very useful

[DHowett]

Conhost has supported mouse tracking for years now! It should support SGR, Xterm and Xterm UTF-8 encodings for Any, Button or (the third common kind that I cannot recall) tracking. vttest will attest to this.

In general, the purpose of this thread is to imagine what terminal emulation should be capable of in the future.

If you are having problems with things that should work today, or your imagination only extends to things that already exist, please file actual bugs so we can track them and prioritize them 😄

[determin1st]

hmm, oke, i was starting my app from cmd.exe and powershell.exe and thought they pass mouse events, seem not. DECRQM will be a brand new shining in chrome feature for the future :]

[determin1st]

another new feature:

resize event, like private mode number 1008 (for example) and the sequence <CSI> columns ; rows M

clarification of the wheel event:

how to extract wheel event from the input? how can terminal user distinguish vertical up from down and horizontal left from right. those arent in the docs anywhere. i found it in the ConEmu source code, me, terminal user who only should read docs, not terminal source code

[Tonal-all]

Like tmux's send-keys feature, I need to send commands to a specific tab using wt, for example: wt send-keys -t window_id:tab_id:pane_id 'pwd\u000d'.

Thanks

[determin1st]

esc-sequence will be more effective, instead of launching new instance only to parse arguments and die. how do you know window_id stuff

[Tonal-all]

Sorry, I think there might be a misunderstanding with the phrase "Like tmux's send-keys feature." What I need from the send-keys feature is the ability to send commands to a shell in a tab, assuming a tab with a shell environment is already running. Using wt, I want to send commands to the shell to achieve functionality similar to remote invocation. If no tab is found, it should start one and execute the subsequent commands. If a tab is found but the shell environment is unavailable, it should still send the commands, and any command execution failure should not be the responsibility of wt.

By the way, Windows Terminal's send-input can only send predefined character sequences, which is not flexible enough to send commands to a shell in a tab.

[determin1st]

why not run shell my-script-file.sh my-arguments-to-the-script, also theres an autoit script engine, it can send keystrokes to window found by title (you still didnt say how you find window_id or tab_id thing). im not windows terminal aware, just dont get why it needs to have that, imo shell should automate itself

[Tonal-all]

Thank you. Inspired by your response, I found a way to replace my feature request. My use case is to use the Neovim editor in WSL to edit files when double-clicking them in Windows File Explorer. For this, I wrote a batch script. However, when directly launching Neovim with the wsl nvim dir/file command, Bash does not load the .bashrc configuration file. I wanted a better method than directly loading .bashrc. So, I started a Windows Terminal window with an available WSL environment. Since I created the window and tab, I know the window_id and tab_id. I thought that if the wt command had a send-keys feature like tmux, it would be more flexible and possible.

Inspired by your response, I opened a WSL tab in Windows Terminal and established a tmux session within it. Now, I can use wsl tmux send-keys other_args in the Windows batch script to accomplish my task.

Thank you again for your kindness and patience.

[OS-Scratch]

You could add a terminal app store so that you could install modules such as Node.js using an UI instead of WinGet.