
Let's say you want to create a system where each server can "join" a network and start exchanging something with other members within that network. How would you approach such a task?
You could create a small centralized orchestrator that will accept join requests and handle that for the recipients, but that's not decentralized. What if we want a truly "plug and play" solution without any centralized servers?
One such solution is federation.
What is federation
A good example would be SMTP (the protocol for sending emails). It's mostly decentralized. Anyone can run their own mail server. There is no single central authority that says "yes! this server can have email, but that one? NOOO!".
Each server communicates using a well-known protocol and exchanges information server-to-server.
You can even call it an "API with known payload schemas", although that is a bit of a simplification.
Let's go with the email thing and start with the obvious. Have you ever considered why email addresses contain an @<domain> part? Why can't it just be my_nick?
It's necessary to know which domain is responsible for that identity! duh!
That domain plays a very important role in the whole federation exchange.
That's because if you write from your email, let's say adam@x.com, to barney@y.com, the sending mail server will take that y.com part, do DNS discovery for MX records, connect to the mail server responsible for that domain, and start exchanging messages.
This is the core idea of federation: servers talk to each other directly after some "discovery process". In our small email example, the discovery part would be a DNS lookup for MX records.
Federated Messenger Protocol
Ok, but we came here to create our own messenger protocol, so let's start.
First, we need to gather some requirements like good system (and in this case, protocol) designers.
Let's assume that we want to have end-to-end encryption - so the server should only see encrypted message data, and end clients will handle encryption and decryption.
We also don't want to blindly trust anyone who sends an HTTP request and says "hello, I am x.com".So we need a way to verify that a federation request was signed by a server that controls the source domain.
Also, we don't want to make it super complicated, so let's drop any fancy things like attachments, multi-device solutions, groups, read receipts, etc. For now, we want to have a decentralized, simple text exchange with only one device. Welcome to the 90s! :D
That should be the basic minimum for our needs. So how do we even start doing it?
Let's start with the easiest, and then move to more complex stuff.
User Identity
Let's also assume that every server already has some users with registration, login, etc. So we don't spend a million years doing everything from scratch.
We need to enforce that user identity will be like an email.
So <nickname>@<domain>, otherwise our federation will not know who to call.
The important thing here is that adam@x.com is not an email address. It only uses the same shape because the shape is useful. The domain tells other servers where to discover federation metadata.
We could go with, for example, x.com > adam, x.com#adam, or adam on x.com. It's your call.
Discovery
Now the first real work. Discovery is the part similar to the MX record lookup process mentioned earlier.
We have full freedom on how we want to handle this, so let's do it with a simple file on the server.
For this, we can use the typical .well-known directory.
In our case, we can call our protocol lnchat (from LearN chat).
This will be our first requirement for the protocol.
Server MUST provide a file under:
GET /.well-known/lnchat.json
Inside we can put the version and other important information that we will craft later in the article. For now, we can go with only the version:
{
"protocol": "lnchat.federation.v1"
}
By the way, it's good practice to always version protocols. It enforces the contract and saves you from a lot of headaches.
With this, servers can check if the servers we want to reach even support our protocol. If yes, they can pull all the necessary information from this one .well-known file.
End to End
Since we want to have end-to-end encryption, we need client-side encryption keys.
Usually that means one private key that stays with the client, and one public key that the user's server can share with other servers.
The public key is used by senders as input to encryption for that user.
The private key is used by the receiver to decrypt it.
The private key MUST NOT be sent to the server as plaintext. If the server can read the private key, then the server can read the messages, and we don't really have end-to-end encryption anymore.
Important: if a server is responsible for publishing users' public keys, a malicious or compromised server could lie about a key. Real end-to-end encryption needs some way to notice that, for example contact key warnings, fingerprints, TOFU, or key transparency.
So we need an endpoint for sharing users' public keys. Every server should implement it as part of our protocol. So let's craft it!
First, let's add information to our discovery file about where to look for the endpoint. This gives every server the ability to choose its own URLs.
It's better not to enforce strict paths here as it may collide with already existing routes on the server.
{
"protocol": "lnchat.federation.v1",
"endpoints": {
"resolve_users": "https://example.com/federation/v1/users/resolve"
}
}
But the resolve_users URL could be a different URL on every server.
Resolving users
The resolve_users endpoint will be responsible for returning current public encryption keys for users from that server.
Other servers will call it before sending a message, so they know which key should be used for which recipient.
We can craft a schema for this endpoint like this:
{
"protocol": "lnchat.federation.v1",
"users": ["adam@x.com", "barney@x.com"]
}
Why an array of users and not just one? Let's imagine that it will be used in high-volume servers. We should offer a way to ease the performance.
If the server can resolve several users in one request, it will be much more performant - fewer HTTP calls.
And for the response, let's do this:
{
"protocol": "lnchat.federation.v1",
"status": "ok",
"users": [
{
"user": "adam@x.com",
"status": "ok",
"key": {
"key_id": "adam-enc-2026-07-a",
"public_key": "base64url-raw-32-byte-x25519-public-key"
}
},
{
"user": "barney@x.com",
"status": "not_deliverable"
}
]
}
So the user can expose a key with a key_id and public_key.
Why do we need a key_id?
Well, imagine how the flow would look like.
The client gets the user's public key and probably stores it in some cache, then sends messages using that key.
What if that key changes? The receiver will never decrypt it. We need to have a way to distinguish keys from each other and even show a warning that the key changed! Maybe it was compromised!
I also added two statuses: ok and not_deliverable. ok is self-explanatory, but I added not_deliverable for cases where we don't have a public key for that user yet.
Remember that the user needs to generate those keys. We cannot safely do it on the backend if we want real end-to-end encryption.
So there is a chance that we already have users who never generated them. We should clearly state that this user is not a valid user for receiving encrypted messages.
Publishing keys
For now, we only handled the endpoint for server-to-server key exchange. But how will that public key appear in the first place?
The user can use the Web Crypto API to generate keys, store the private key locally, and send the public key to the backend to keep it in the DB.
const keyPair = await window.crypto.subtle.generateKey(
{ name: "X25519" },
false,
["deriveBits"]
);
const publicKey = await window.crypto.subtle.exportKey(
"raw",
keyPair.publicKey
);
Generally, we should avoid exporting the private key outside of the client unless we really need to. Browsers can store CryptoKey objects in IndexedDB.
If the private key is non-extractable, it is harder to accidentally leak it as a string somewhere.
This key is not literally used to encrypt the text directly. It is used by the encryption method to create a shared secret between sender and receiver.
So how would the flow look from the user's perspective?
Someone Else's ServerHome ServerIndexedDBClientSomeone Else's ServerHome ServerIndexedDBClientgenerate encryption keysstore private keystore public_keyget .well-known/lnchat.jsoncall resolve_usersuser_public_key
Here we assume that our home server can store the public_key, and it will be exposed through the resolve_users endpoint according to our protocol schema.
Messaging
Now we can resolve the recipient's public key. Great. But we still need to actually send something.
For the first version, let's keep the responsibility split simple:
The client encrypts the message body.
The home server validates the request, creates the federation envelope, and sends it to the recipient server.
The recipient server gets the message, validates it, and stores it until the user requests it.
The user decrypts messages using their private key.
Recipient ClientRecipient ServerSender ServerSender ClientRecipient ClientRecipient ServerSender ServerSender Clientask for recipient public keyresolve_userspublic key + key_idpublic key + key_idencrypt message bodyencrypted messagefederation messageencrypted messagedecrypt message body
First, the client encrypts the message locally.
Then the client could send something like this to its own server:
{
"recipient": "barney@y.com",
"recipient_key_id": "barney-enc-2026-07-a",
"encapsulated_key": "base64url-hpke-enc",
"ciphertext": "base64url-ciphertext"
}
We will talk about the encapsulated_key and ciphertext in a minute. So for now let's just accept that it's a "secret message".
And the plaintext inside the ciphertext can be very small:
{
"protocol": "lnchat.message.v1",
"type": "text",
"body": "hello"
}
Or the actual message could be simple text, but it's better to pack it in some sort of format so we can send more metadata alongside the message itself. Maybe later this could be used for a simple version of attachments, etc.
The server then creates its own HTTP request and sends it to the Recipient Server. But to which endpoint?
Well, to achieve that we need to add one more endpoint to our discovery file.
{
...
"endpoints": {
...
"send_messages": "https://example.com/federation/v1/message"
}
}
Now Sender Server can take the encryption data and send it to Recipient Server.
Payload can look like this:
{
"protocol": "lnchat.federation.v1",
"message_id": "019f47a9-b138-79d2-8ec3-38a2a9326955",
"sender": "adam@x.com",
"recipient": "barney@y.com",
"recipient_key_id": "barney-enc-2026-07-a",
"encapsulated_key": "base64url-hpke-enc",
"ciphertext": "base64url-ciphertext"
}
Notice that we added message_id and sender, but they were not present in the client message. This is intentional. We shouldn't trust the client to provide the real sender id. The user is already authenticated, so we can always put their name there.
And message_id is there for idempotency. If the server sends the same message twice, and the first one was already processed, we should reject the second one.
The message_id should be a randomly generated string. We can go with UUIDv7, for example.
recipient_key_id should be the same id we get from the resolve_users endpoint. It tells the recipient which key was used to encrypt the message.
So how to encrypt payload?
What's up with the encapsulated_key, you may ask?
Well, for a modern messaging system it's good to use something called HPKE.
HPKE itself is a bit too heavy to explain fully in this post, so let's use a simpler mental model.
Encrypting directly with asymmetric keys can be computationally heavy. So the idea is:
We generate a new symmetric key. This is the kind of key where the same key encrypts and decrypt
Texto de karboosx.net tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en karboosx.net · Volver a la noticia de El Autómata