The motivation
--------------

When deploying applications and dependencies to customer VMs, it is
useful to reduce the number of files and the total size you need to
upload.

While we could use something like gzip for this, it would be better to
use zstd because it manages to compress better and faster. And even
after deployment, we can use it in our tools and scripts.

As the result of this project, I want to have a single executable file
that I can copy to a VM to compress and decompress data. I don't want
to deal with glibc version incompatibilities.

And I don't want to need root access to install any dependencies, this
should work on any VM we are given, regardless of permissions.

The solution
------------

A good candidate for doing this is musl libc, which is designed to be
statically compiled. Alpine Linux uses musl libc by default, and makes
this really easy to do. We can just use a Podman container and build a
static zstd binary inside it.

Aside from working on zstd-compressed files, the zstd tool can also use
the lz4, gz and xz/lzma formats. This makes it a pretty good
"universal" compression tool, and having all that in a single binary is
very convenient.

Below is the script I used to compile a static zstd binary on Alpine
Linux.

    apk update
    apk add wget
    apk add make
    apk add gcc
    apk add musl-dev

    apk add lz4-dev
    apk add lz4-static
    apk add zlib-dev
    apk add zlib-static
    apk add xz-dev
    apk add xz-static

    ZSTD_VERSION=1.5.4
    URL="https://github.com/facebook/zstd/releases/download"
    URL="${URL}/v${ZSTD_VERSION}/zstd-${ZSTD_VERSION}.tar.gz"

    wget -O zstd.tar.gz "${URL}"
    tar -xzf zstd.tar.gz
    cd zstd-${ZSTD_VERSION}

    export CFLAGS="-flto"
    export LDLAGS="-flto -static -static-libgcc -static-libstdc++"
    export HAVE_LZ4=1
    export HAVE_ZLIB=1
    export HAVE_LZMA=1
    make zstd-release

    cp programs/zstd /output/path/zstd


The results
-----------

And the result? A single 2.6 MB file that can compress/decompress any
format we care about using.

$ file ./zstd
./zstd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically
linked, with debug_info, not stripped

$ ls -lah ./zstd
-rwxrwxrwx 1 root root 2.6M Mar 21 13:28 ./zstd

$ ./zstd --help

  --format=zstd          Compress files to the `.zst` format. [Default]
  --format=gzip          Compress files to the `.gz` format.
  --format=xz            Compress files to the `.xz` format.
  --format=lzma          Compress files to the `.lzma` format.
  --format=lz4           Compress files to the `.lz4` format.


--
Leo