Professor Steven J. Murdoch

What’s new in EMVLab

2017-06-22T09:41:55+01:00

I set up EMVLab back in 2009, along with my former colleague, Mike Bond. We set out to provide useful tools for people working in payment systems development and research, and based on feedback from users I think broadly speaking it’s done its job. I’ve kept it up and running, but there have been some features which are now showing their age.

In the past few months I’ve moved the site to a dedicated virtual machine in UCL (resolving some reliability and performance problems), but this also gave me an opportunity to update some other aspects that I’ve long been putting off. This post summarises these updates, as well as some of my future plans.

HTTPS availability

EMVLab is now available over HTTPS, thanks to Let’s Encrypt. At least for now I am not re-directing HTTP to HTTPS or using HTTPS-enforcing measures like HSTS. This is because one of the principles behind EMVLab was that it should require the bare minimum in terms of browser capability. Workstations in some payment systems companies are incredibly locked-down and so features like Javascript might not be available. I don’t know of any specific instances, but I could imagine that HTTPS is blocked in some cases too.

Basic handling of 3-byte tags

Many data structures in EMV are represented in tag-length-value format (part of the ASN.1 standard) and one of the most popular tools in EMVLab is the TLV decoder. When I set up the website all tags were one or two bytes long, and this assumption was built into the decoder. Since then the contactless EMV standards have now come out, and these do include 3-byte tags and so users of the site were reporting that the these broke the decoder.

I’ve now generalised the TLV decoder to handle arbitrary length tags (as per the specification), so such tags will not break the decoder. However I still have not updated the tag list so it will only describe tags present in EMV 4.1. Other tags will be shown and properly decoded, but identified as unknown tags. I hope to later update the list of tags to include more recent EMV tags, including contactless variants.

Basic handling of unknown tags

Related to the above, the EMV tag database will now generate a description for unknown tags. Obviously there’s not much that can be said about tags not in the database, but there is some structure in tag values as defined in the ASN.1 specification so I can tell whether the tag is constructed or primitive, and its class. For some tag ranges the EMV specification also states who can define new tags. These results are now shown.

Future plans

As mentioned above, I do plan to update the tag database. I also hope to incorporate some other helpful feedback I’ve received from users but have not yet had time to address. If you have particular requests please do get in touch either by email or the contact form.

Fixing slow startup of mc on Mac OS X using dtruss

2015-08-04T10:06:39+01:00

I sometimes use Midnight Commander (mc) as a file manager. It’s widely available, is text based so works over SSH, it’s quite capable, and it’s very fast. However on Mac OS X something has changed (I’m not sure exactly what) that results in mc 4.8.14 taking about 10 seconds to start. If you just want to fix this problem skip to the end, or read on for how I found the solution.

A great tool for debugging all sorts of problems on Mac OS X is dtruss, which prints out details of the system calls a program makes. dtruss itself is just a script using the powerful DTrace framework, but if dtruss does what you need it’s a lot easier to use than writing your own DTrace script.

Normally to debug program you would just run dtruss program and look at the output. However using dtruss to debug interactive text-mode program problems is a bit tricky. The first problem is that dtruss, like all DTrace scripts, needs root privilege to run (and mc’s problem goes away when run as root). The second problem is that the output of dtruss will be obscured by the ncurses interface of mc. These could be solved by running dtruss in a different (root) terminal then attaching it to the mc process (running as a normal user), but attaching to a process requires knowing the PID and by the time dtruss is attached, mc would have finished starting up.

The trick is to attach dtruss to another program, get dtruss to follow child processes as they are started (the -f flag), and have this other program wait until dtruss has attached before starting mc. A simple shell script that achieves the goal is:

#!/usr/local/bin/bash
echo "PID" $$
read
mc

To attach dtruss, first run the shell script and note the PID of bash that it prints out. Then, in a different terminal running as root, attach dtruss to bash (i.e. sudo dtruss -af -p PID). Finally, allow mc to start by pressing enter in the terminal that’s running the shell script. In the terminal running dtruss you can then see the system calls of particular interest (before and after the long startup delay).

Here’s a snippet of the output, where ... is where the delay happens.

18458/0xc8907:   1197089      400       5 select(0x6, 0x7FFF5E716660, 0x0, 0x0, 0x7FFF5E716650)      = 1 0
18458/0xc8907:   1197092        3       1 read(0x3, "bash: PROMPT_COMMAND: line 1: syntax error near unexpected token `;'\r\nbash: PROMPT_COMMAND: line 1: `update_terminal_cwd; ; pwd>\0", 0x80)        = 128 0
18458/0xc8907:   1197094        3       1 select(0x6, 0x7FFF5E716660, 0x0, 0x0, 0x7FFF5E716650)      = 1 0
18458/0xc8907:   1197095        2       0 read(0x3, "&6;kill -STOP $$'\r\n\0", 0x80)         = 19 0
18458/0xc8907:   1196289   605871  604986 fork()         = 18459 0
...
18458/0xc8907:   1197142 10001050      37 select(0x6, 0x7FFF5E716660, 0x0, 0x0, 0x7FFF5E716650)      = 0 0
18458/0xc8907:   1197196       53       1 sigaction(0x2, 0x7FFF5E7166D8, 0x0)        = 0 0
18458/0xc8907:   1197247       18      14 write_nocancel(0x1, "\033[?1001s\033[?1002h\033[?1006hread_nocancel\0", 0x18)      = 24 0

The columns are PID/thread, relative time (µs), elapsed time (µs), CPU time (µs) and syscall. The time elapsed for the last select call is 10 seconds, which explains the startup delay. This select seems to be related to reading from a file descriptor that contains an error message involving “bash”, “PROMPT_COMMAND” and “update_terminal_cwd”.

While it’s starting up, mc creates a subshell, and the error message seems to be related to that. Perhaps because of the error message, mc fails to detect that the subshell has properly started up and only continues when it falls-back by disabling subshell support.

Looking for the strings involved in the error message turns up a function defined in /etc/bashrc:

# Tell the terminal about the working directory at each prompt.
if [ "$TERM_PROGRAM" == "Apple_Terminal" ] && [ -z "$INSIDE_EMACS" ]; then
    update_terminal_cwd() {
        # Identify the directory using a "file:" scheme URL,
        # including the host name to disambiguate local vs.
        # remote connections. Percent-escape spaces.
        local SEARCH=' '
        local REPLACE='%20'
        local PWD_URL="file://$HOSTNAME${PWD//$SEARCH/$REPLACE}"
        printf '\e]7;%s\a' "$PWD_URL"
    }
    PROMPT_COMMAND="update_terminal_cwd; $PROMPT_COMMAND"
fi

Something is going wrong with update_terminal_cwd() and the if condition suggests that running inside Emacs is a problem so it looks likely that mc has the same problem. To prevent nested subshells, mc sets the the MC_SID environment variable so this is a suitable check for disabling update_terminal_cwd inside the mc subshell.

;tldr

So the fix is to change the line in /etc/bashrc:

if [ "$TERM_PROGRAM" == "Apple_Terminal" ] && [ -z "$INSIDE_EMACS" ]; then

if [ "$TERM_PROGRAM" == "Apple_Terminal" ] && [ -z "$INSIDE_EMACS" ] && [ -z "$MC_SID" ]; then

and now mc starts near instantaneously.

Lost frog

2015-07-28T14:15:03+01:00

This frog was trapped by my basement and I took this photo before taking it to a better home in the garden. Taken using a Fstoppers Flash Disc to soften the light; you can just see its outline in the eye.