This week I've been focussing on adding Terminal Emulation into the project and spent some time experimenting with the various terminal widgets available. The winner in this instance is "xterm.js".

Getting it up and running was a snip, and it looks pretty cool, and over websockets, you wouldn't really know it wasn't a local xterm, but there turned out some be some very time consuming gotcha's.

The Font of all problems

Yes Google fonts look great, and yes if you run xterm.js with Google's Ubuntu Mono font it looks brilliant, but it doesn't work with Unicode so as soon as you run anything that's not pure ASCII you're starting to ask for trouble - it's not worth it. Instead use "Lucida Console", it's built-in, just as good, and won't screw around with things like "pstree".

Getting the right signals

If one just spawns a process, it turns around that sending a SIGINT to the process won't do what you want because the signal won't get propagated as expected to subprocesses. In English, Control+C isn't going to work for you. So, the first thing you need do when the process spawns is to run run setsid, then you need to connect the process to a pty device to make it think it's connected to a terminal and hence can do interactive stuff. There is a lovely little hack in the POpen call which was made for the job;

self.master, self.slave = pty.openpty()
self.process = subprocess.Popen(
            ["bash","-c","-l"],
            preexec_fn=os.setsid,
            shell=True,
            stdin=self.slave,
            stdout=self.slave,
            close_fds=True,
            stderr=subprocess.STDOUT)
Changes in the Matrix ...

The next thing you notice is that as soon as you resize your browser, or any component related to your terminal, stuff breaks. So you need to attach a resize event to the the xterm.js fit addon, which will resize the browser Window. All well and good, but if you run top you'll still find it sitting in an 80x24 window in the corner of your screen. Solution; you need to use an ioctl to tell the shell session that the screen size has changed. Moreover, once you've done this, you need to tell any running program (top for example) that the screen dimensions have changed and it needs a geometry reload. The latter is done by sending a SIGWINCH signal to the process concerned. As this process will be started by the POpened shell you won't have a reference to it, but the blanket solution is to send a signal to the process group of the process that you DO know about, which should propagate to all relevant parties. Here's a nice bit of working resize code;

def set_size(self, row, col, xpix=0, ypix=0):
        """set the shell window size"""
        winsize = struct.pack("HHHH", row, col, xpix, ypix)
        fcntl.ioctl(self.slave, termios.TIOCSWINSZ, winsize)  
        pgrp = os.getpgid(self.process.pid)
        os.killpg(pgrp,signal.SIGWINCH)  
And I just hate Unicode

This one was a bit of a brain-teaser. When reading from the back-end terminal session, the maximum buffer size on the pipe is 4096 bytes, so if you do something that generates a lot of output (pstree is my test tool here), your response will be fed back through the system as a bunch of packets with a max-size of 4096. But here's the rub, Unicode characters take up more than a single byte position, so if you're randomly splitting up a large data stream containing Unicode characters, there's a reasonable chance you'll chop a Unicode character in half, which will result in a fairly terminal Unicode error as soon as you try to use the result. Here's my solution, if anyone has something better I'm all ears - in the meantime, this works .. :)

def get(self,channel):
        """get data from an channel, or None if we're done"""
        sock = self.master
        while not self.finished:
            r,w,e = select.select([sock],[],[],1)
            if sock not in r: continue
            bytes = ''
            while not self.finished:
                try:
                    bytes += os.read(sock, 4096)
                    bytes.decode('utf-8')
                    return bytes
                except UnicodeDecodeError:
                    pass

Essentially, ensure we have a valid unicode string before returning, otherwise, read the next block and test the two joined together, etc ...

Anyway, it all looks like it's working now, so the next step is to move it from a tab into a Gridster dashboard tool. The tools have built-in resizers, I'm idly wondering whether I can dynamically change the font size inside the terminal window as the tool is resized ...

Update :: current state of play;