shinGETsu is a federated Textboard software used in Japan.
Almost all the discussion currently on the network is in Japanese.
2024-12-28
Introduction
Crescent operates through the cooperation of multiple nodes, implying the existence of an underlying protocol. We refer to this as the primary protocol. This document attempts to formalize and redefine the primary protocol, but unfortunately, it is not yet complete.
The Crescent Protocol version defined in this document is 0.7. The protocol consists of the following four layers:
Communication Layer
Crescent Network Structure
The Crescent network is formed by connections between nodes. Each node has a unique name on the network, called the node name. Each node on the network stores the names of other nodes to communicate with them. The node names stored by a node for network maintenance are called neighboring nodes. It is desirable, but not required, that neighboring nodes mutually store each other’s node names.
Communication between nodes conforms to the HTTP/1.0 or HTTP/1.1 specifications. Request and response messages between nodes must follow the GET method conventions.
For convenience, node responses are considered plain text with UTF-8 character encoding. Strictly speaking, the responses are structured, and the character encoding is defined for each part.
Crescent communication commands between nodes are ultimately constructed as URLs. Commands sent to other nodes must comply with URL specifications.
Node
A node refers to a single unit of a program or computer implementing the Crescent Protocol. Nodes connecting to the Crescent network must accurately implement the Crescent Protocol. Implementations of the Crescent Protocol in programs must also comply with HTTP and URL specifications.
Each node has a unique name on the network, called the node name. The format of the node name is defined as follows:
hostname:port_number/pathname
Here, the hostname is a DNS name, an IPv4 IP address, or an IPv6 IP address enclosed in square brackets ([]). The port number is an integer in decimal representation.
The Crescent Protocol does not specify the port number, a component of the node name. Each node can freely configure the port number used in its node name. If the port number used in the node name is less than 1024, it should conform to the well-known port numbers.
Message
Information exchanged between nodes during request and response is called a message. Message content varies widely. For example, a response to a /ping command contains the IP address of the communicating node. Files exchanged with the /get command are also considered messages.
Nodes sending request messages to and receiving response messages from other nodes should be able to interpret messages compressed in gzip format in addition to regular uncompressed messages. However, when sending a gzip-compressed message to a node, the sending node must analyze the message header of the request to determine whether the receiving node can interpret gzip-compressed messages. If, after this determination, the node is capable of interpreting gzip-compressed messages, it should send the message in gzip format. Nodes capable of interpreting compressed messages should, whenever possible, notify other nodes of their capability. Compressing messages offers the benefits of reduced bandwidth usage.
Time
The base time used in the Crescent Protocol is “January 1, 1970, 00:00:00 (GMT).” A time is represented as an integer value of seconds relative to the base time.
Protocol Commands between Nodes
Communication between nodes is performed by sending protocol commands. Protocol commands are appended to the end of the path portion of the URL generated from the node name. An example of a URL with a protocol command appended is:
http://example.com:8000/server.cgi/ping
The following are the protocol commands for establishing communication between nodes and the behavior of the receiving node. \n represents a newline character (0x0a).
/ping: The node returns “PONG\nIP address of the other node”.
/node: The node selects one node it is connected to and returns its node name.
/join/node_name: The node decides whether to add the other node as a neighboring node. If so, it returns “WELCOME” or “WELCOME\nNode name of another node”. The decision might involve verifying the validity of the provided node name using /ping, checking if the DNS name portion of the node name points to the connecting node itself, etc., but these are not defined here. The hostname in the other node’s node name can be omitted. The path in the node name should have ‘/’ replaced with ‘+’. The requesting node expects to be added to the receiving node’s list of maintained nodes and to connect to the node specified in the response.
/bye/node_name: The node removes the other node from its neighboring node list and returns “BYEBYE”. The decision might involve checking if the DNS name portion of the node name points to the connecting node itself, etc., but these are not defined here.
/have/filename: Returns “YES” if the node has the file specified by filename, otherwise “NO”.
/get/filename/time_argument: Returns the records of the file specified by filename that satisfy the time_argument. The time_argument is one of the following:
"time": Record at the specified time.
"-time": All records before the specified time.
"time-": All records after the specified time.
"time-time": All records between the specified times.
"time/identifier": Record with the specified time and identifier.
/head/filename/time_argument: Same response as /get, but returns only the time and identifier for each record.
/update/filename/time/identifier/node_name: Informs the node that the file has been updated. If the node is already processing a file update, it does nothing. If the node has the file, it updates it and then notifies its connected nodes, replacing the node_name with its own. Otherwise, it forwards the received protocol command to its connected nodes. The path in the node_name should have ‘/’ replaced with ‘+’. Similar to /join, the hostname can be omitted.
/recent/time_argument: Returns the names of the records that received /update commands within the range specified by time_argument as a pseudo-file. The time_argument is one of the following:
"time": Record at the specified time.
"-time": All records before the specified time.
"time-": All records after the specified time.
"time-time": All records between the specified times.
The response format is:
timestamp<>identifier<>filename
Each line can have an added string called a tag, in which case the format becomes:
timestamp<>identifier<>filename<>tag:tag_group
A tag_group is a space-separated list of tags. Tags cannot contain spaces, <, >, or &.
File Propagation
When a file’s content changes, a node can notify other nodes of the change using the /update command. The process for determining whether to notify about file changes varies depending on the implementation.
File
A file is a collection of zero or more records that constitute a single bulletin board. These records are connected by \n characters. A file has infinite length, and in practice, the entirety of a file cannot be viewed. Nodes always handle a portion of a file.
Record
A record is an element constituting a file. The format of a record is defined as follows:
timestamp<>identifier<>body
The identifier is the MD5 value of the body. The elements of a record are delimited by <>. The timestamp represents the number of seconds elapsed since the base time as an integer.
Intermediate Layer
Filename
The filename format is prefix_basename. Allowed characters for prefix are alphanumeric characters, and allowed characters for basename are alphanumeric characters and underscores (_).
The prefix represents the type of file. The basename will likely represent the “name” of the file, etc. The application defines the filename.
Body Format
Elements constituting the body are delimited by “<>”. The body consists of multiple named fields. The format of a named field is “name:value”. Allowed characters for named field names are alphanumeric characters and underscores (_). Named field names must not be duplicated. Exception handling for duplicated names is implementation-dependent. The named fields stamp and id are reserved, representing the timestamp and identifier, respectively. The order of appearance of named fields in the body does not matter. Field values of named fields must not contain “<” or “>”. As an exception, they can contain <string> (called a tag) (the string must have a length of 1 or more characters and must not contain “<” or “>”). The application defines which tags can be used.
Signature
Refer to Crescent Protocol 0.5 Draft #2 PDF for the definition of signatures.
Signature-Related Named Fields
Deletion Notification-Related Named Fields
Implementations that share the communication layer but have different intermediate layers can be created. Such implementations are referred to as “communication layer compatible with Crescent 0.7”.
Application Layer
Plugins and Applications
A plugin is a concept that includes applications, but it cannot yet be defined. Perhaps a plugin is everything other than communication, signatures, and deletion notifications.
Among plugins, those that define the file format are called applications. The Crescent Protocol 0.7 defines only one application: thread. There used to be 8 other applications. Note that an application is not the same as an implementation.
Implementations can define applications other than thread. Such implementations are referred to as “intermediate layer and below compatible with Crescent 0.7”.
Thread Application
This is a typical bulletin board application.
Filename
The filename format is “thread_encoded_board_title”. The encoded board title is a string obtained by representing the board title in UTF-8 and converting it to hexadecimal representation (0-9A-F).
Named Fields
name: Poster’s name.
body: The content of the post. Only the “
” tag, representing a newline, can be used. “
” and “
” are not allowed.
attach: Base64-encoded attached file.
suffix: File extension of the attached file.
mail: Poster’s email address.
Application Group
This is a standard for multiple applications to cooperate with each other.
Wiki-style Namespace Application Group
Currently, only thread is defined, but if applications are created according to the following standard, they can cooperate with thread.
Filename
The filename format is “application_name_encoded_board_title”. The encoded board title is a string obtained by representing the board title in UTF-8 and converting it to hexadecimal representation (0-9A-F).
Bracket Links
When an application converts the content of a file into some data format and outputs the result, if there are bracket links as described below in the body, the application must create links specified by the string.
Assume that an application type exists. The standard bracket link format is [[/type/string]]. The format of the string is defined by the application (type). /type can be omitted. In the case of a bracket link with /type omitted, [[string]], it is considered that the type of the file format where the body is recorded is omitted.
Examples of creating bracket links for thread are shown below:
[[thread]]: If the file format where the body is recorded is thread, create a link to the thread titled “thread”.
[[thread/7889e7db]]: If the file format where the body is recorded is thread, create a link to the record with identifier “7889e7db” in the thread titled “thread”.
[[/thread/thread/7889e7db]]: Create a link to the record with identifier “7889e7db” in the thread titled “thread”.
Please provide comments: [[Crescent Development]]