Show HN: A pure bash web server. No netcat, socat, etc. (github.com)

what is the difference in these two approaches?

I always use the pipe, it is intuitive to me to think left-to-right, output of this becomes input of next. But I always see stackoverflow answers using the other approach.

    while :; do <page.html nc -l 80 ; done

Pretty printed: https://gist.github.com/tyingq/4e568425e2e68e6390f3105e58878...

The /inet/tcp/8080/0/0 is a gawk-ism, versus a Linux thing:

https://www.gnu.org/software/gawk/manual/html_node/TCP_002fI...

Unfortunately, gawk doesn't let you control the listen/accept part of it, so it doesn't scale well. Not sure why they did it that way. A separate accept() call would have made it actually usable in a non-toy way.

Thanks! TIL! That's a much cleaner way of doing it.

Regarding keeping the file, if memory serves the "real" pipeline had some `tee`s in it. I recreated this much more recently to make a GitHub repo of funny code snippets (https://github.com/MaxBondABE/one-thread-crashing/).

You reminded me of a different toy I made many years ago. I created FIFOs for files that you might try to read when exploiting an arbitrary file read in a web app, say /etc/www/apache.conf (if that's the right path - it's been a while since I've configured a web server!) and I'd have a program that opened them. This would block until someone opened the other end of the FIFO, at which point I'd raise an alarm (which was just a print statement).

I'm not OP, but as they said, it's an HTTP proxy. Run it, then point your browser at http://localhost:8080. The message is proxied over to http://httpbin.org (on port 80, the normal HTTP port). If you're in an overly restricted environment and have a legit need to proxy to a very limited set of hosts, you can ad-hoc make a proxy mapping from local port to a single specific remote host. Just change the 8080, destination host, and buffer file.

And I mean this as encouragement, but you should read the synopsis from the GNU netcat manual. It's short and explains it well enough. netcat httpbin.org 80 is using connect mode, the -l one is using listen mode, and -p is --local-port (the port to listen on in listen mode).

Of course!

A.)

Some context:

- Our goal here is to match up to two network connections, so that anything we receive on one is forwarded to the other.

- netcat is a "networking swiss army knife", it's a tool for making network connections

- HTTP and bash pipelines both happen to be "line oriented", meaning they operate one line at a time. Exploiting this happy coincedence is what makes this work

- httpbin.org is a metanym for any website you need.

So:

    echo "" > /tmp/buffer

Initialize our buffer to an empty state

    tail -f /tmp/buffer

Read each line in the buffer, one at a time, and feed them to the next command. Do this forever (-f).

    netcat httpbin.org 80

Connect to the website httpbin.org. Transmit each line from our buffer to the website.

    netcat -lp 8080 > /tmp/buffer

Accept a connection. Transmit any data we receive from httpbin.org to this client; write any data we receive from the client to the buffer, so that it can be transmitted to httpbin.org.

B.) I don't think there is one! I've never needed to do this again in the, gosh, seven intervening years. Normally bash pipelines operate like a bucket brigade, moving data unidirectionally through various processing steps, and usually dumping it into a file at the end.

They aren't normally for creating long-lived programs that manage bidirectional flows of data. Which is why writing a web server in bash is a fun challenge.

This is like saying my Python app is written in C because Python is C.

A single-threaded server is fine if you're only mildly IO bound, which is the case when you're serving files-as-resources. My understanding anyway is that http.server is explicitly not for production use!

Parent does point out they use this for quick tinkering.

Will Apple ditch Python in the future? Typing on an M1 Air and I`m pretty sure Python came per-installed.

Why does it matter if it's included, it's trivial to install and extremely broadly useful?

> Something that's not true on Windows.

Getting python up an running on Windows is fairly trivial though. You don't even need local admin permissions if your machine is locked down by your IT dept.

To spare you the trap you can open the file as FD in your bash process (e.g., exec {my_fd}>"$TMPFILE") and then directly delete it before doing anything else, and use the /proc handle to access it ("/proc/$$/fd/${my_fd}")

  true | sleep infinity &
  STDINPID="$!"
  TMPPIPE="/proc/$STDINPID/fd/0"
  echo hello > "$TMPPIPE"
  echo write more > "$TMPPIPE"
  cat "$TMPPIPE"
  echo write again > "$TMPPIPE"
  cat "$TMPPIPE"
  kill -9 "$STDINPID" # clean up pipe

  true | sleep infinity &
  STDINPID="$!"
  exec {mypipe}<"/proc/$STDINPID/fd/0" # hijack FD
  TMPPIPE="/proc/$$/fd/${mypipe}"
  disown "$STDINPID" # suppress killed message on stderr
  kill -9 "$STDINPID" # clean up process now already
  echo hello > "$TMPPIPE"
  echo write more > "$TMPPIPE"
  cat "$TMPPIPE"
  echo write again > "$TMPPIPE"
  cat "$TMPPIPE"

  exec {checkfd}>/dev/null
  CHECKFDPATH="/proc/$$/fd/${checkfd}"
  (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) &
  STDINPID="$!"
  WRITER="/proc/$STDINPID/fd/1"
  
  while IFS= read -r LINE; do
    # only echo lines if we didn't close the connection yet
    if [ -e "$CHECKFDPATH" ]; then
      echo "$LINE" | tr '[:lower:]' '[:upper:]' > "$WRITER"
      if [ "$LINE" = "bye" ]; then
        exec {checkfd}<&-
      fi
    else
      echo "received while closed: $LINE"
    fi
  done < <(nc -q 1 -l 8080 < "$WRITER")
  if [ -e "$CHECKFDPATH" ]; then
    # close checkfd to close writer pipe
    exec {checkfd}<&-
  fi

  exec {checkfd}>/dev/null
  CHECKFDPATH="/proc/$$/fd/${checkfd}"
  (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) &
  STDINPID="$!"
  disown "$STDINPID"
  READER="/proc/$STDINPID/fd/1"
  
  { while IFS= read -r LINE; do
    echo "$LINE" | tr '[:lower:]' '[:upper:]'
    if [ "$LINE" = "bye" ]; then
      echo exiting > /dev/stderr
      break
    fi
  done < "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } | { nc -q 1 -l 8080 > "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; }
  exec {checkfd}<&-

can use $RANDOM to get more entropy/avoid collisions, though as you said, better not to use files at all.

I get it. The challenging part is fun. You can learn all the corner cases, the clever bash-isms, and then show off. But the code you end up with is essentially unmaintainable, especially since you don't have any test coverage, so...

This is my favorite "reference" bash script: https://git.einval.com/cgi-bin/gitweb.cgi?p=abcde.git;a=blob...

...it feels like "if it's possible to do in bash, this does it".

BATS is

1) bloated

2) doesn't use a very nice syntax (hint: bashisms are not a very nice syntax). Compare BATS assertions with this for example (the test of my test library, which uses the library's own functions): https://github.com/pmarreck/tinytestlib/blob/yolo/test Call me crazy but I think `assert_equal "4" "$result"` is a lot more readable than `[ "$result" -eq 4 ]`

3) doesn't let you assert on all of (return code, stdout, stderr) from running a single command. I created a `capture` function to do that and bring those all into named variables that you can then assert on.

But overall I wanted something tiny that you could easily include alongside a bash script instead of establishing as a new dependency you'd have to install separately. Honestly, I feel that functions like assert_equal, assert_not_equal, assert_success, assert_failure, assert_match and assert_no_match (which all raise on failure) should be built into every programming language's standard library.

I know that you can sort of get by just using `set -e` and just do a comparison operation, but...

Including the somewhat unexpected "Bottoms Up Bash - Colon Cancer Prevention Project".

true is also a POSIX utility (not builtin), so it can be used in POSIX-y sh scripts as well.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

This prints abridged usage help, though. You can use `info bash read`, but confusingly it might not always contain the same content as manpages.

Thanks I didn't know! So many years of searching for "read" (etc) in the huge bash man page and, of course, in the case of "read" finding many false positives. Still it would be nice if `man` could be improved to know about builtin commands - the obvious thing is something like an html anchor in the big bash man page for each built in.