05 October 2023

NTP client

I used to sync my computer’s time using a script called httptime. httptime is a script that fetches the current date and time by making HTTP requests and reading the Date response header.

This worked pretty well, and I’d still consider it a good solution. But I also wanted to use what everyone else uses for time sync, which is NTP (Network Time Protocol).

I read through the RFCs, and it seems pretty simple to use. You send one UDP packet to an NTP server, you get another packet back. And that packet contains the current timestamp.

To pick a random NTP server, I used the following code snippet.

HOST = "pool.ntp.org"
PORT = 123

resolve = socket.getaddrinfo(HOST, PORT, socket.AF_INET, socket.SOCK_DGRAM)

addresses = []

for row in socket.getaddrinfo(HOST, PORT, socket.AF_INET, socket.SOCK_DGRAM):

host, port = random.choice(addresses)

This just runs a DNS query, gets a bunch of NTP servers and picks one randomly.

The protocol itself is just sending a 48-byte packet and receiving a 48-byte packet in response. The packets are in the same format, you just send an empty packet and receive one that is filled in.

10 October 2023

MUID miner rewrite in Rust

When I started playing with Microprediction, I wrote a simple MUID miner in Python. I used multiprocessing to parallelize the work, and while it wasn’t super fast, it was fast enough to get me started.

If you’re not familiar with them, I wrote a blog post about MUIDs last year.

Rewriting this in a faster language like Rust or C was on my todo list for a while, and I finally got around to it.

The crates I used were sha256 v1.4.0 and crossbeam v0.8.2. SHA256 is because the keys we mine need to have specific prefixes when hashed with SHA256, and crossbeam is because I like its channel API.

The Python version used very simple data structures, I stored the prefixes of valid keys as strings in a set. Determining the “difficulty” of a key was done by getting the length of the prefix. This worked fine, but it was overall not the most efficient way to do it.

In the Rust version, I converted the prefixes to 64-bit integers, and stored the difficulty in the least significant 4 bits. After that, all the prefixes were stored in a hash set.

Another speed gain was in generating the keys. In the Python version, I was getting random bytes from /dev/urandom and converting them to hex. This resulted in a system call, and we really didn’t need cryptographically secure randomness.

In the Rust version; I read some bytes from /dev/urandom for each miner thread, and then incremented the bytes by 1 for each key. This was much faster.

Something else that made the Rust version more convenient to use was the include_str! macro. This allowed me to bundle the data file that contained the animal prefixes with the binary, and resulted in a single executable that could be copied easily.

Writing the initial version in Python was definitely the right choice. And coming back for a version 2 in Rust meant that I had an idea of what I needed, and a reference implementation to compare against.