netcat to 👀 peep HTTP: gotchas

netcat is an old tool sometime qualified as the swiss army knife of network utilities.

That was what I remembered as I wondered how I could capture HTTP traffic between my reverse proxy and one of my service on my snowflake. After a quick internet research, I found several source citing command similar to this one:

mkfifo answer
touch in out
nc -vv -l -k -p 8001 <answer | tee -a in /dev/fd/2 | nc service.com 80 | tee -a out answer

Command explanation

It’s quite a lengthy command. Let’s decompose this:

• mkfifo answer: Make a named pipe¹. In essence the same thing than the regular unix pipe |, except it exists on the filesystem thus you reference it by its name.

• touch in out: Create files named in and out, respectively for persisting the streams of incoming request and outgoing responses.

• nc -vv -l -k -p 8001 <answer:

  netcat² in listen mode will print incomming request content to stdout and send stdin back to client.
  It listens to tcp port 8001, then reads value of named pipe answer to send it back to the client.
  -k enable netcat to continue listening after connection close (netcat can accept several successive connections)
  -vv enable logs

• tee -a in /dev/fd/2:

  tee will read stdin to write back to stdout and files in parameters.
  stdin here being content of HTTP requests, it will write it to in file and to /dev/fd/2 (which is stderr, to see request content in console). Implicitly tee will also print to stdout

• nc service.com 80:

  Connects to service.com on port 80 and write stdin content on the socket. Server response is then written on stdout.
  stdin here being HTTP request content which is chained up until here.

• tee -a out answer: You know what tee does now. Note that we write back server response to the answer named pipe.

Everything from the first nc is chained through unix pipes | which redirects content of left hand side stdout to right hand side stdin.
If you have difficulties wrapping your head around it, try playing with individual commands like nc, tee, read about unix pipes and named pipes.

Possible issues

Which flavor of netcat?!

Alright, netcat has different implementations, namely openbsd and gnu. Examples in this article assumes you have openbsd netcat.

Knowing which implementation of netcat you use is important; several options are behaving differently. Use `nc -h` to know which version you use and what does the options do.

On Ubuntu, both versions exists with packages netcat-openbsd and netcat-traditional. Don’t forget to sudo update-alternatives --config nc to use appropriate implementation.

Additionnaly, you have a similar tool named `ncat` which comes from `nmap` project. It has interesting options such as `-o` which dump a netcat session to a file, `-k` which makes the listening socket accept several connections and `-c` which execute command on incomming connection. Actually, `ncat` is more suited to our task since we can achieve our little peeking proxy with this command:

ncat -vv -l -k -p 8001 -c "ncat service.com 80" -o dump

Server closes the connection

Our approach works… but it’s quite fragile.

In fact, the program which connects to distant server to communicate quits when connection ends, it does not reconnect automatically.
If connection to server.com ends, our chain is broken and incomming requests to 8001 will just hang.

In order to observe this behavior, you can use header Connection: close on your HTTP request. Full example:

$ nc -vv -l -k -p 8001 <answer | tee -a in /dev/fd/2 | nc google.com 80 | tee -a out answer & # Google will do fine for testing
$ http localhost:8001 # using httpie
# Request is OK
$ http localhost:8001
# Still OK
$ http localhost:8001 "Connection: close"
# Still OK, although server just closed the connection, any other request will hang
$ http localhost:8001
# NOK, Request hangs

You can use ss command to observe opened socket throughout the testing

watch ss  -ap 'sport == :8001 or dport == :8001 or sport == :80 or dport == :80' # poor man monitoring of socket on 8001 and 80

It doesn’t work on fish shell

If you are using fish shell you may have noticed that nc -vv -l -k -p 8001 <answer | tee -a in /dev/fd/2 | nc google.com 80 | tee -a out answer doesn’t even start a listening socket (connection refused on localhost:8001).

This is because, contrary to bash, fish tries to open redirected files in the parent process, before forking different part of the pipeline, resulting in a deadlock.

One can rewrite the command to avoid redirected files in order to be compliant with fish:

cat answer | nc -vv -l -k -p 8001 | tee -a in /dev/fd/2 | nc google.com 80 | tee -a out answer

I will let you explore this stackoverflow thread for more details.