VT100 Terminal Emulation in Windows WPF or Silverlight

Question

I'm thinking of creating a WPF or Silverlight application that acts like a terminal window. In addition, since it is in WPF / Silverlight, it will be able to “improve” the terminal experience with effects, images, etc.

I am trying to find a better way to imitate a terminal. I know how to handle VT100 emulation before parsing, etc. But how to display it? I looked at using RichTextBox and essentially converting VT100 escape codes to RTF.

The problem I'm seeing is performance. A terminal can only receive a few characters at a time, and in order to be able to load them into a textbox like-we-go, I constantly created TextRanges and used Load () to load the RTF. In addition, in order for each “session” download to be completed, it would have to fully describe RTF. For example, if the current color is Red, RTF codes are needed for each load in the TextBox so that the text is red, or I believe that RTB will not load it as red.

This seems very redundant - the resulting RTF document created by the emulation will be extremely messy. In addition, the carriage movement does not look like it will be perfectly processed by RTB. I need something custom says, but it scares me!

Hoping to hear bright ideas or pointers to existing solutions. Perhaps there is a way to insert the actual terminal and overlays on top of it. The only thing I found is the old WinForms control.

UPDATE: See how the proposed solution crashes due to perfection in my answer below. :(
VT100 terminal emulation in Windows WPF or Silverlight

Answer 1

If you try to implement this with RichTextBox and RTF, you will quickly come across many limitations and find that you spend much more time working on differences than if you implemented your functions yourself.

It’s actually quite simple to implement VT100 terminal emulation using WPF. I know, because now I have implemented an almost complete VT100 emulator in an hour or so. To be precise, I have included everything except:

• Keyboard input,
• Alternative character sets,
• A few esoteric VT100 modes that I have never seen

The most interesting were:

• Double-wide / double-height characters for which I used RenderTransform and RenderTransformOrigin
• Blinking, for which I used animation for a shared object, so all characters will blink together
• Underlined for which I used Grid and Rectangle to make it look more like a VT100 display
• Cursor and selection, for which I set a flag in the cells themselves and use DataTriggers to change the display
• Using both a one-dimensional array and a nested array pointing to the same objects, making scrolling and selection easier

Here is the XAML:

<Style TargetType="my:VT100Terminal"> <Setter Property="Template"> <Setter.Value> <ControlTemplate TargetType="my:VT100Terminal"> <DockPanel> <!-- Add status bars, etc to the DockPanel at this point --> <ContentPresenter Content="{Binding Display}" /> </DockPanel> </ControlTemplate> </Setter.Value> </Setter> </Style> <ItemsPanelTemplate x:Key="DockPanelLayout"> <DockPanel /> </ItemsPanelTemplate> <DataTemplate DataType="{x:Type my:TerminalDisplay}"> <ItemsControl ItemsSource="{Binding Lines}" TextElement.FontFamily="Courier New"> <ItemsControl.ItemTemplate> <DataTemplate> <ItemsControl ItemsSource="{Binding}" ItemsPanel="{StaticResource DockPanelLayout}" /> </DataTemplate> </ItemsControl.ItemTemplate> </ItemsControl> </DataTemplate> <DataTemplate DataType="{x:Type my:TerminalCell}"> <Grid> <TextBlock x:Name="tb" Text="{Binding Character}" Foreground="{Binding Foreground}" Background="{Binding Background}" FontWeight="{Binding FontWeight}" RenderTransformOrigin="{Binding TranformOrigin}"> <TextBlock.RenderTransform> <ScaleTransform ScaleX="{Binding ScaleX}" ScaleY="{Binding ScaleY}" /> </TextBlock.RenderTransform> </TextBlock> <Rectangle Visibility="{Binding UnderlineVisiblity}" Height="1" HorizontalAlignment="Stretch" VerticalAlignment="Bottom" Margin="0 0 0 2" /> </Grid> <DataTemplate.Triggers> <DataTrigger Binding="{Binding IsCursor}" Value="true"> <Setter TargetName="tb" Property="Foreground" Value="{Binding Background}" /> <Setter TargetName="tb" Property="Background" Value="{Binding Foreground}" /> </DataTrigger> <DataTrigger Binding="{Binding IsMouseSelected}" Value="true"> <Setter TargetName="tb" Property="Foreground" Value="White" /> <Setter TargetName="tb" Property="Background" Value="Blue" /> </DataTrigger> </DataTemplate.Triggers> </DataTemplate> 

And here is the code:

 public class VT100Terminal : Control { bool _selecting; static VT100Terminal() { DefaultStyleKeyProperty.OverrideMetadata(typeof(VT100Terminal), new FrameworkPropertyMetadata(typeof(VT100Terminal))); } // Display public TerminalDisplay Display { get { return (TerminalDisplay)GetValue(DisplayProperty); } set { SetValue(DisplayProperty, value); } } public static readonly DependencyProperty DisplayProperty = DependencyProperty.Register("Display", typeof(TerminalDisplay), typeof(VT100Terminal)); public VT100Terminal() { Display = new TerminalDisplay(); MouseLeftButtonDown += HandleMouseMessage; MouseMove += HandleMouseMessage; MouseLeftButtonUp += HandleMouseMessage; KeyDown += HandleKeyMessage; CommandBindings.Add(new CommandBinding(ApplicationCommands.Copy, ExecuteCopy, CanExecuteCopy)); } public void ProcessCharacter(char ch) { Display.ProcessCharacter(ch); } private void HandleMouseMessage(object sender, MouseEventArgs e) { if(!_selecting && e.RoutedEvent != Mouse.MouseDownEvent) return; if(e.RoutedEvent == Mouse.MouseUpEvent) _selecting = false; var block = e.Source as TextBlock; if(block==null) return; var cell = ((TextBlock)e.Source).DataContext as TerminalCell; if(cell==null) return; var index = Display.GetIndex(cell); if(index<0) return; if(e.GetPosition(block).X > block.ActualWidth/2) index++; if(e.RoutedEvent == Mouse.MouseDownEvent) { Display.SelectionStart = index; _selecting = true; } Display.SelectionEnd = index; } private void HandleKeyMessage(object sender, KeyEventArgs e) { // TODO: Code to covert e.Key to VT100 codes and report keystrokes to client } private void CanExecuteCopy(object sender, CanExecuteRoutedEventArgs e) { if(Display.SelectedText!="") e.CanExecute = true; } private void ExecuteCopy(object sender, ExecutedRoutedEventArgs e) { if(Display.SelectedText!="") { Clipboard.SetText(Display.SelectedText); e.Handled = true; } } } public enum CharacterDoubling { Normal = 5, Width = 6, HeightUpper = 3, HeightLower = 4, } public class TerminalCell : INotifyPropertyChanged { char _character; Brush _foreground, _background; CharacterDoubling _doubling; bool _isBold, _isUnderline; bool _isCursor, _isMouseSelected; public char Character { get { return _character; } set { _character = value; Notify("Character", "Text"); } } public Brush Foreground { get { return _foreground; } set { _foreground = value; Notify("Foreground"); } } public Brush Background { get { return _background; } set { _background = value; Notify("Background"); } } public CharacterDoubling Doubling { get { return _doubling; } set { _doubling = value; Notify("Doubling", "ScaleX", "ScaleY", "TransformOrigin"); } } public bool IsBold { get { return _isBold; } set { _isBold = value; Notify("IsBold", "FontWeight"); } } public bool IsUnderline { get { return _isUnderline; } set { _isUnderline = value; Notify("IsUnderline", "UnderlineVisibility"); } } public bool IsCursor { get { return _isCursor; } set { _isCursor = value; Notify("IsCursor"); } } public bool IsMouseSelected { get { return _isMouseSelected; } set { _isMouseSelected = value; Notify("IsMouseSelected"); } } public string Text { get { return Character.ToString(); } } public int ScaleX { get { return Doubling!=CharacterDoubling.Normal ? 2 : 1; } } public int ScaleY { get { return Doubling==CharacterDoubling.HeightUpper || Doubling==CharacterDoubling.HeightLower ? 2 : 1; } } public Point TransformOrigin { get { return Doubling==CharacterDoubling.HeightLower ? new Point(1,0) : new Point(0,0); } } public FontWeight FontWeight { get { return IsBold ? FontWeights.Bold : FontWeights.Normal; } } public Visibility UnderlineVisibility { get { return IsUnderline ? Visibility.Visible : Visibility.Hidden; } } // INotifyPropertyChanged implementation private void Notify(params string[] propertyNames) { foreach(string name in propertyNames) Notify(name); } private void Notify(string propertyName) { if(PropertyChanged!=null) PropertyChanged(this, new PropertyChangedEventArgs(propertyName)); } public event PropertyChangedEventHandler PropertyChanged; } public class TerminalDisplay : INotifyPropertyChanged { // Basic state private TerminalCell[] _buffer; private TerminalCell[][] _lines; private int _height, _width; private int _row, _column; // Cursor position private int _scrollTop, _scrollBottom; private List<int> _tabStops; private int _selectStart, _selectEnd; // Text selection private int _saveRow, _saveColumn; // Saved location // Escape character processing string _escapeChars, _escapeArgs; // Modes private bool _vt52Mode; private bool _autoWrapMode; // current attributes private bool _boldMode, _lowMode, _underlineMode, _blinkMode, _reverseMode, _invisibleMode; // saved attributes private bool _saveboldMode, _savelowMode, _saveunderlineMode, _saveblinkMode, _savereverseMode, _saveinvisibleMode; private Color _foreColor, _backColor; private CharacterDoubling _doubleMode; // Computed from current mode private Brush _foreground; private Brush _background; // Hidden control used to synchronize blinking private FrameworkElement _blinkMaster; public TerminalDisplay() { Reset(); } public void Reset() { _height = 24; _width = 80; _row = 0; _column = 0; _scrollTop = 0; _scrollBottom = _height; _vt52Mode = false; _autoWrapMode = true; _selectStart = 0; _selectEnd = 0; _tabStops = new List<int>(); ResetBuffer(); ResetCharacterModes(); UpdateBrushes(); _saveboldMode = _savelowMode = _saveunderlineMode = _saveblinkMode = _savereverseMode = _saveinvisibleMode = false; _saveRow = _saveColumn = 0; } private void ResetBuffer() { _buffer = (from i in Enumerable.Range(0, Width * Height) select new TerminalCell()).ToArray(); UpdateSelection(); UpdateLines(); } private void ResetCharacterModes() { _boldMode = _lowMode = _underlineMode = _blinkMode = _reverseMode = _invisibleMode = false; _doubleMode = CharacterDoubling.Normal; _foreColor = Colors.White; _backColor = Colors.Black; } public int Height { get { return _height; } set { _height = value; ResetBuffer(); } } public int Width { get { return _width; } set { _width = value; ResetBuffer(); } } public int Row { get { return _row; } set { CursorCell.IsCursor = false; _row=value; CursorCell.IsCursor = true; Notify("Row", "CursorCell"); } } public int Column { get { return _column; } set { CursorCell.IsCursor = false; _column=value; CursorCell.IsCursor = true; Notify("Row", "CursorCell"); } } public int SelectionStart { get { return _selectStart; } set { _selectStart = value; UpdateSelection(); Notify("SelectionStart", "SelectedText"); } } public int SelectionEnd { get { return _selectEnd; } set { _selectEnd = value; UpdateSelection(); Notify("SelectionEnd", "SelectedText"); } } public TerminalCell[][] Lines { get { return _lines; } } public TerminalCell CursorCell { get { return GetCell(_row, _column); } } public TerminalCell GetCell(int row, int column) { if(row<0 || row>=Height || column<0 || column>=Width) return new TerminalCell(); return _buffer[row*Height + column]; } public int GetIndex(int row, int column) { return row * Height + column; } public int GetIndex(TerminalCell cell) { return Array.IndexOf(_buffer, cell); } public string SelectedText { get { int start = Math.Min(_selectStart, _selectEnd); int end = Math.Max(_selectStart, _selectEnd); if(start==end) return string.Empty; var builder = new StringBuilder(); for(int i=start; i<end; i++) { if(i!=start && (i%Width==0)) { while(builder.Length>0 && builder[builder.Length-1]==' ') builder.Length--; builder.Append("\r\n"); } builder.Append(_buffer[i].Character); } return builder.ToString(); } } ///////////////////////////////// public void ProcessCharacter(char ch) { if(_escapeChars!=null) { ProcessEscapeCharacter(ch); return; } switch(ch) { case '\x1b': _escapeChars = ""; _escapeArgs = ""; break; case '\r': Column = 0; break; case '\n': NextRowWithScroll();break; case '\t': Column = (from stop in _tabStops where stop>Column select (int?)stop).Min() ?? Width - 1; break; default: CursorCell.Character = ch; FormatCell(CursorCell); if(CursorCell.Doubling!=CharacterDoubling.Normal) ++Column; if(++Column>=Width) if(_autoWrapMode) { Column = 0; NextRowWithScroll(); } else Column--; break; } } private void ProcessEscapeCharacter(char ch) { if(_escapeChars.Length==0 && "78".IndexOf(ch)>=0) { _escapeChars += ch.ToString(); } else if(_escapeChars.Length>0 && "()Y".IndexOf(_escapeChars[0])>=0) { _escapeChars += ch.ToString(); if(_escapeChars.Length != (_escapeChars[0]=='Y' ? 3 : 2)) return; } else if(ch==';' || char.IsDigit(ch)) { _escapeArgs += ch.ToString(); return; } else { _escapeChars += ch.ToString(); if("[#?()Y".IndexOf(ch)>=0) return; } ProcessEscapeSequence(); _escapeChars = null; _escapeArgs = null; } private void ProcessEscapeSequence() { if(_escapeChars.StartsWith("Y")) { Row = (int)_escapeChars[1] - 64; Column = (int)_escapeChars[2] - 64; return; } if(_vt52Mode && (_escapeChars=="D" || _escapeChars=="H")) _escapeChars += "_"; var args = _escapeArgs.Split(';'); int? arg0 = args.Length>0 && args[0]!="" ? int.Parse(args[0]) : (int?)null; int? arg1 = args.Length>1 && args[1]!="" ? int.Parse(args[1]) : (int?)null; switch(_escapeChars) { case "[A": case "A": Row -= Math.Max(arg0??1, 1); break; case "[B": case "B": Row += Math.Max(arg0??1, 1); break; case "[c": case "C": Column += Math.Max(arg0??1, 1); break; case "[D": case "D": Column -= Math.Max(arg0??1, 1); break; case "[f": case "[H": case "H_": Row = Math.Max(arg0??1, 1) - 1; Column = Math.Max(arg0??1, 1) - 1; break; case "M": PriorRowWithScroll(); break; case "D_": NextRowWithScroll(); break; case "E": NextRowWithScroll(); Column = 0; break; case "[r": _scrollTop = (arg0??1)-1; _scrollBottom = (arg0??_height); break; case "H": if(!_tabStops.Contains(Column)) _tabStops.Add(Column); break; case "g": if(arg0==3) _tabStops.Clear(); else _tabStops.Remove(Column); break; case "[J": case "J": switch(arg0??0) { case 0: ClearRange(Row, Column, Height, Width); break; case 1: ClearRange(0, 0, Row, Column + 1); break; case 2: ClearRange(0, 0, Height, Width); break; } break; case "[K": case "K": switch(arg0??0) { case 0: ClearRange(Row, Column, Row, Width); break; case 1: ClearRange(Row, 0, Row, Column + 1); break; case 2: ClearRange(Row, 0, Row, Width); break; } break; case "?l": case "?h": var h = _escapeChars=="?h"; switch(arg0) { case 2: _vt52Mode = h; break; case 3: Width = h ? 132 : 80; ResetBuffer(); break; case 7: _autoWrapMode = h; break; } break; case "<": _vt52Mode = false; break; case "m": if (args.Length == 0) ResetCharacterModes(); foreach(var arg in args) switch(arg) { case "0": ResetCharacterModes(); break; case "1": _boldMode = true; break; case "2": _lowMode = true; break; case "4": _underlineMode = true; break; case "5": _blinkMode = true; break; case "7": _reverseMode = true; break; case "8": _invisibleMode = true; break; } UpdateBrushes(); break; case "#3": case "#4": case "#5": case "#6": _doubleMode = (CharacterDoubling)((int)_escapeChars[1] - (int)'0'); break; case "[s": _saveRow = Row; _saveColumn = Column; break; case "7": _saveRow = Row; _saveColumn = Column; _saveboldMode = _boldMode; _savelowMode = _lowMode; _saveunderlineMode = _underlineMode; _saveblinkMode = _blinkMode; _savereverseMode = _reverseMode; _saveinvisibleMode = _invisibleMode; break; case "[u": Row = _saveRow; Column = _saveColumn; break; case "8": Row = _saveRow; Column = _saveColumn; _boldMode = _saveboldMode; _lowMode = _savelowMode; _underlineMode = _saveunderlineMode; _blinkMode = _saveblinkMode; _reverseMode = _savereverseMode; _invisibleMode = _saveinvisibleMode; break; case "c": Reset(); break; // TODO: Character set selection, several esoteric ?h/?l modes } if(Column<0) Column=0; if(Column>=Width) Column=Width-1; if(Row<0) Row=0; if(Row>=Height) Row=Height-1; } private void PriorRowWithScroll() { if(Row==_scrollTop) ScrollDown(); else Row--; } private void NextRowWithScroll() { if(Row==_scrollBottom-1) ScrollUp(); else Row++; } private void ScrollUp() { Array.Copy(_buffer, _width * (_scrollTop + 1), _buffer, _width * _scrollTop, _width * (_scrollBottom - _scrollTop - 1)); ClearRange(_scrollBottom-1, 0, _scrollBottom-1, Width); UpdateSelection(); UpdateLines(); } private void ScrollDown() { Array.Copy(_buffer, _width * _scrollTop, _buffer, _width * (_scrollTop + 1), _width * (_scrollBottom - _scrollTop - 1)); ClearRange(_scrollTop, 0, _scrollTop, Width); UpdateSelection(); UpdateLines(); } private void ClearRange(int startRow, int startColumn, int endRow, int endColumn) { int start = startRow * Width + startColumn; int end = endRow * Width + endColumn; for(int i=start; i<end; i++) ClearCell(_buffer[i]); } private void ClearCell(TerminalCell cell) { cell.Character = ' '; FormatCell(cell); } private void FormatCell(TerminalCell cell) { cell.Foreground = _foreground; cell.Background = _background; cell.Doubling = _doubleMode; cell.IsBold = _boldMode; cell.IsUnderline = _underlineMode; } private void UpdateSelection() { var cursor = _row * Width + _height; var inSelection = false; for(int i=0; i<_buffer.Length; i++) { if(i==_selectStart) inSelection = !inSelection; if(i==_selectEnd) inSelection = !inSelection; var cell = _buffer[i]; cell.IsCursor = i==cursor; cell.IsMouseSelected = inSelection; } } private void UpdateBrushes() { var foreColor = _foreColor; var backColor = _backColor; if(_lowMode) { foreColor = foreColor * 0.5f + Colors.Black * 0.5f; backColor = backColor * 0.5f + Colors.Black * 0.5f; } _foreground = new SolidColorBrush(foreColor); _background = new SolidColorBrush(backColor); if(_reverseMode) Swap(ref _foreground, ref _background); if(_invisibleMode) _foreground = _background; if(_blinkMode) { if(_blinkMaster==null) { _blinkMaster = new Control(); var animation = new DoubleAnimationUsingKeyFrames { RepeatBehavior=RepeatBehavior.Forever, Duration=TimeSpan.FromMilliseconds(1000) }; animation.KeyFrames.Add(new DiscreteDoubleKeyFrame(0)); animation.KeyFrames.Add(new DiscreteDoubleKeyFrame(1)); _blinkMaster.BeginAnimation(UIElement.OpacityProperty, animation); } var rect = new Rectangle { Fill = _foreground }; rect.SetBinding(UIElement.OpacityProperty, new Binding("Opacity") { Source = _blinkMaster }); _foreground = new VisualBrush { Visual = rect }; } } private void Swap<T>(ref T a, ref T b) { var temp = a; a = b; b = temp; } private void UpdateLines() { _lines = new TerminalCell[Height][]; for(int r=0; r<Height; r++) { _lines[r] = new TerminalCell[Width]; Array.Copy(_buffer, r*Height, _lines[r], 0, Width); } } // INotifyPropertyChanged implementation private void Notify(params string[] propertyNames) { foreach(string name in propertyNames) Notify(name); } private void Notify(string propertyName) { if(PropertyChanged!=null) PropertyChanged(this, new PropertyChangedEventArgs(propertyName)); } public event PropertyChangedEventHandler PropertyChanged; } 

Please note: if you do not like the visual style, just update the TerminalCell DataTemplate. For example, the cursor may be a blinking rectangle rather than a solid one.

This code was interesting for writing. Hope this will be helpful for you. He probably has a mistake or two (or three) as I have never done it, but I expect them to be easy to clear up. I would welcome editing this answer if you fix something.

Answer 2

I don’t understand why you will worry about RTF getting confused. Yes it will. But it’s not your burden to handle this, a Microsoft programmer did this a while ago, having to write code to render a confusing RTF. It works well and is completely opaque to you.

Yes, it will not be super-fast. But hey, you are emulating an 80x25 display that used to run at 9600 baud. Completely replacing the control in order to try to make it optimal does not make much sense and will become a serious matter.

Answer 3

Well, to report my status, I decided that this was not real with WPF or Silverlight .

The problem with the proposed approach is that there are 80 * 24 TextBlocks plus some other elements with several bindings for forecolor, backcolor, etc. When the screen needs to scroll, each of these bindings needs to be reevaluated and it is very, very slow. Updating the entire screen takes a few seconds. In my application this is unacceptable, the screen will constantly scroll.

I tried many different things to optimize it. I tried using one text block with 80 passages per line. I tried to supplement change notifications. I tried to make the scroll event manually update each text block. Nothing helps - the slower part is updating the user interface, not how it is done.

One thing that could help was that I developed a mechanism to not have a text block or work for each cell, but only to change text blocks when changing the text style. Thus, a line of one text color, for example, will be only 1 text block. Nevertheless, it would be very difficult, and, in the end, it would help scenarios that change little on the screen. My application will have many colors flying (I think ANSI art), so in this case it will still be slow.

Another thing that I thought of will help if I don't update the text blocks, but scroll them while the screen scrolls. Thus, the text blocks will move from top to bottom, and then only new ones will need to be updated. I managed to get this work using an observable collection. It helped, but its STILL WAY TOO SLOW!

I even reviewed the WPF user control using OnRender. I created one that used drawingContext.RenderText in various ways to figure out how fast it could be. But EVEN THAT is too slow to constantly refresh the screen.

So this .. I abandoned this design. Instead, I look at using a real console window, as described here:

No console output from a WPF application?

I don’t really like this, since the window is separate from the main window, so I’m looking for a way to embed a console window in a WPF window, if possible. I will ask another question about how to do this and will link it here when I do this.

UPDATE: Turning on the console window also failed because it does not require the title bar to be removed. I implemented it as a low-level WinForms user control and I host it in WPF. This works great and after some optimizations very quickly.

Answer 4

The only way to effectively display text is to use TextFormatter. I have implemented a telnet client for text RPG games and it works very well. You can check sources at http://mudclient.codeplex.com