Limited touch support on mobile devices impacts terminal usability

[friuns2]

Describe the bug

The terminal has limited support for touch interactions on mobile devices and touch screens. This severely impacts usability on mobile devices as users cannot effectively interact with the terminal using touch gestures and mobile-specific input methods.

To Reproduce

1. Open xterm.js terminal on a mobile device or touch screen
2. Try to interact with the terminal using:
   • Touch gestures for scrolling
   • Pinch-to-zoom
   • Touch selection of text
   • Long press for context menu
   • Double tap for word selection
3. Note that these interactions either don't work or fall back to basic mouse event translation

Expected behavior

The terminal should provide comprehensive touch support with:

1. Native touch event handling instead of mouse event translation
2. Support for common touch gestures:
   • Single tap for cursor positioning
   • Double tap for word selection
   • Long press for context menu
   • Two-finger pinch for zoom
   • Two-finger scroll for terminal scrolling
3. Touch-optimized text selection with:
   • Selection handles
   • Magnifying glass for precise cursor positioning
   • Mobile-friendly context menu
4. Proper integration with virtual keyboards and mobile keyboard accessories

Current behavior

• Touch interactions are limited or non-functional
• Mobile experience relies on browser's mouse event translation
• No native touch gesture support
• Text selection is difficult on touch devices
• Virtual keyboard integration is basic
• No mobile-optimized interface elements

Screenshots

No screenshots available - this is a feature limitation report

Environment

• OS: iOS 16+, Android 12+
• Browser: Safari iOS, Chrome Android, Firefox Android
• Device type: Mobile phones, tablets, touch-enabled laptops
• xterm.js version: Latest

Additional context

This affects all mobile and touch-enabled devices. Current implementation in CoreBrowserTerminal.ts focuses primarily on mouse and keyboard events, with no dedicated touch event handling.

Proposed Solution

1. Implement TouchHandlingService for native touch event processing
2. Add touch-specific configuration options to ITerminalOptions
3. Enhance CoreBrowserTerminal with touch event bindings
4. Add mobile-optimized UI elements for selection and interaction
5. Improve virtual keyboard integration

See technical implementation details in the previous description for complete solution architecture.

[jerch]

@friuns2 Well the gist of mobile device support - it is non-existent (beside what the mobile browsers try to achieve by limited automapping of events and stuff). Our whole frontend handling is very desktop centered, made for a standard keyboard and a mouse. Even our CI-testing does only probe for typical desktop stuff.

We basically have a blind spot regarding mobile devices and their interaction pattern. This is unlikely to change as long as no one steps up and tries to fill in the missing bits. I doubt that anyone of us will have the time to do that, I for myself dont even have enough knowledge to tackle that (well I am not that much on the frontend here anyway). I guess @Tyriar prolly wont have time to do that either.

So plz feel free to get something rolling towards better mobile support. But also be warned upfront - parts of the key handling is very old and tricky code, that evolved over time from issue reports and cannot be tested in a straight forward way. I'd call it the ugly duck in xterm.js, where touching anything is likely to introduce new issues at various non-obvious ends. A nightmare to rework.

[bajrangCoder]

Actually, there aren’t any built-in touch related stuffs except that double tap , word selection, etc, and integrating with native components isn’t really possible (as far as I know). In one of my projects where I use Xterm on a touch device, I implemented all the touch interactions myself. Selection is handled through custom selection handles and by calculating touch coordinates. I also created custom selection handles and context menus, which makes the experience much more usable ,though it’s not perfect, it’s a significant improvement over the default behavior. There are still some quirks, especially due to mobile virtual keyboards, but overall, this approach works well. I believe a similar method could be applied to Xterm to make it more touch-friendly.

Video

InShot_20250724_190232145.mp4

[Tyriar]

Related: #1101

For touch support, using the default DOM renderer and/or screenReaderMode would be a requirement to get touch working. I'm not sure the state of that though.

For input, AFAIK you basically need to implement your own keyboard or at least some bar with all the relevant characters. I don't know if you can give a hint to the browser to disable how mobile IMEs collect characters and instead have them send each keystroke immediately. This is out of scope for this library.

Feel free to contribute improvements to touch support (the smaller the PRs the better), but this is definitely not a priority for me.