Decentralized net services

a.k.a. web3, DEX, P2P, Peer-to-peer, friend-to-friend; Internet for an untrustworthy world

September 18, 2017 — April 4, 2024

computers are awful together
concurrency hell
confidentiality
distributed
diy
game theory
P2P
Figure 1

Services that run without a server; Peer-to-peer (“P2P”) services. When you combine P2P services with P2P networks you have the internet for the post-industrial surveillance world.

What follows are some miscellaneous peer-to-peer/friend-to-friend/federated forms/self-hosted services for people who worry that their favourite online service might be politically censored by state or corporate manipulation or just dickheads.

Perhaps you want communications to function without a single point of failure. Or maybe you just want freedom from centralised surveillance of your social connections. Presumably most of us would like those things to some extent. The question is if we wish to pay the various convenience costs of not using a corporate-backed state-approved product.

Ideally, I’d like to find robust ways of participating in the internet bidirectionally, in non-real-time, without assuming the internet is plugged in and working, and without requiring all participants to be massive geeks.

Related: the unidirectional offline internet and sneakernets, and low bandwidth hacks, DIY internet infrastructure. Note that chat clients, social media and file syncing are covered elsewhere, which biases what remains. If you want to do this truly securely, a hardened os might be advisable.

Contrast self-hosted, federated and P2P options. By “self-hosted” I mean when I run my own server for my own stuff. In federated systems there are still servers, but anyone can run one for a community and these community servers collaborate for some kind of global coverage. In P2P every client is also a server, and therefore everyone is peers. Further taxonomizing is uninteresting for my current purpose.

Some of these options can be made easier to set up if one used a VPN from the house, maybe even a cheeky mesh-vpn system like tailscale.

There seem to be two priorities that developers bring to making peer-to-peer networks.

  1. Implement a P2P application
  2. Reinvent the internet as an intrinsically P2P system

1 P2P apps

You run an app on your computer which finds peers on the internet somehow, and does something with those peers. The most famous example is probably BitTorrent (for downloading and verifying data), or the Bitcoin blockchain (for arranging financial transactions).

1.1 BitTorrent

A classic.

I have not much to say here, except to note that because the BitTorrent protocol is penalised and blocked in many places, it benefits from an integrated VPN. docker-transmission-openvpn is one elegant solution for that.

1.2 Radicle

radicle is a serverless setup for git, which is already a peer-to-peer system as far as source code management goes — but sometimes you want to coordinate with a few more affordances.

They have a kind of git GUI called upstream. Currently, like many decentralised apps they have a weird permissions model, where everything is public, which is a weird kind of cryptographic security. 🚧TODO🚧 clarify

1.3 Bisq

Bisq is a decentralised network for currency exchange via bitcoins.

1.4 OpenBazaar

Openbazaar is a decentralised network for selling anything, including currency.

1.5 Retroshare

Retroshare is…

  • A Decentralized Social Sharing Network designed For the People with no dependencies on any corporate system or central servers.
  • Make strong crypto useable for the average computer user, have strong crypto in daily communication.
  • The right to have secrets. You decide who can receive information. Hide information from intelligence agencies and spying companies.
  • Freedom of speech, no censorship.
  • Independence from corporate systems and centralized servers. Central services might shut down or change their terms of services at any time. Do you remember Myspace? Or German Studivz? Remember when Facebook changed their terms of service? Skype being bought by Microsoft?
  • Free software. Only free software can provide free and secure communication. Developers can read Retroshare’s source code and verify the security.

Features chat, voice and video, offline mail, file sharing, distributed search, forums and compatibility with TOR. See more info under DIY socials.

1.6 Beaker

Beaker is based on the dat protocol for data sharing. It sounds like zeronet but a bit less shaky, if still not actually well designed for real use cases.

Create and fork websites directly from your computer. You control your data. No blockchain required.

They also have pointed blockchain-mania snark:

We quickly ruled out the blockchain variants (Bitcoin, Ethereum, Namecoin) as the core network model of Beaker for the following reasons:

  • Proof-of-work and global ledger distribution are inefficient. So far, blockchain networks have struggled with throughput as they scale. Betting on a blockchain includes the risk that throughput may decrease over time, and/or that users will be forced to download large datasets to participate.
  • The requirement to use global consensus on all transactions results in a direct cost for operations in the form of transaction fees. We know that many network operations do not require global consensus, and therefore a blockchain makes a poor default for applications.
  • We believe Proof-of-work is wasteful, and we have ethical reservations about the expenditure of resources to sustain PoW. At this time, Proof-of-stake is still an unproven alternative, but we are interested in its potential.
  • We have concerns over governance models. By investing decision-making in miners, the blockchain networks have given power to the entities with the most mining resources. […]

Rhetorical entertainment value aside, whilst I do find this model elegant, it works, for now, only for rich nerds with desktop computers and time to replace their nice corporatised internet with an ugly experimental broken DIY internet. It doesn’t work on smartphones, so the people with the most tenuous internet will not use it and it will not take over the world. I don’t think the founders would claim otherwise; they are all about proof-of-concept. Perhaps some future iteration might change stuff?

update has this been absorbed into the hypercore thing?

1.7 Magic wormhole

The minimal viable sorta-decentralised anything, magic wormhole simply transfers a file from one machine to another, via magic passphrase (which must be transferred separately e.g. by encrypted chat)

Sender:

$ wormhole send README.md
Sending 7924 byte file named 'README.md'
On the other computer, please run: wormhole receive
Wormhole code is: 7-crossover-clockwork
Sending (<-10.0.1.43:58988)..
100%|=========================| 7.92K/7.92K [00:00<00:00, 6.02MB/s]
File sent.. waiting for confirmation
Confirmation received. Transfer complete.

Receiver:

% wormhole receive
Enter receive wormhole code: 7-crossover-clockwork
Receiving file (7924 bytes) into: README.md
ok? (y/n): y
Receiving (->tcp:10.0.1.43:58986)..
100%|===========================| 7.92K/7.92K [00:00<00:00, 120KB/s]
Received file written to README.md

Elegant, no? Tragically it doesn’t yet work on mobile platforms, only desktop.

1.8 Bridgefy chat

A messaging client built as the flagship app of the Bridgefy mesh network, about which I know little except that it doesn’t require the internet, using a P2P system over Bluetooth.

1.9 Bugout

Bugout by Chris Mccormick is a miniature decentralised web proof-of-concept based purely in the browser.

Build back-end web services that run in a browser tab.

  • Host backend services without a VPS, domain or SSL cert.
  • Anyone can deploy by simply opening a browser tab.
  • Can be “self-hosted” by leaving a browser tab open on a PC.
  • Client-server over WebRTC instead of HTTPS.

This is fostering a little ecosystem including rpi bugout boxes.

1.10 UUCP

Classic unix maildrop distributed syncing via UUCP is having a resurgence. See the Dataforge UUCP for some of that.

2 New internet protocols

The whole internet is terrible! Let us make one that is less terrible! There is a collective action problem in getting people to switch to such networks, which anyone who has tried to persuade peers not to Facebook them will appreciate. There is also an incentive problem in getting people to contribute resources; So if you are using a distributed anonymous system then presumably everyone is storing chunks of other people’s data. How do you incentivise that? Some kind of verification that you are pulling your weight? Some kind of blockchain?

2.1 web3

Idk. Robin Sloan turns some good phrases though: “Tokenize all the things”, ”universe of wallets”.

2.1.1 Fleek

The easiest way to build and deploy scalable web3 apps. Fleek is an open source, blockchain agnostic, extensible web3 development platform.

2.2 Scuttlebutt

The flagship social internet-as-social-network thing. See its section under DIY social networks.

2.3 hypercore

The Dat protocol has pivoted to some new protocol now called Hypercore Protocol. I do not know much about this yet. Hacker news argues about this: Hypercore protocol: a distributed (P2P) append-only log.

  • datdot.org/

    DatDot team is building an autonomous hosting network for p2p data systems. Think of it as a Filecoin but for Hypercore protocol, built with Substrate.

2.4 IPFS

Amber Case argues for IPFS.

This project claims to be building an actual decentralised, internet-collapse-ready web-replacement, by getting us to all store files for one another. It is sometimes claimed to be web3 and other times not. Still geeky and badly explained, but an actual ecosystem is reportedly growing around it with ties to the blockchain community. The latter is due I suppose to the “complementary” protocol filecoin, a blockchain incentivization scheme for storing files. The Infuria intro is helpfully concrete.

The IPFS rationale is

The average lifespan of a web page is 100 days. Remember GeoCities? The web doesn’t anymore. It’s not good enough for the primary medium of our era to be so fragile.

IPFS keeps every version of your files and makes it simple to set up resilient networks for mirroring of data.

Developing world. Offline. Natural disasters. Intermittent connections. All trivial compared to interplanetary networking. The networks we’re using are so 20th Century. We can do better.

IPFS powers the creation of diversely resilient networks which enable persistent availability with or without Internet backbone connectivity.

One interesting service is the decentralised databases such as orbitdb. If I were trying to publish my blog on IPFS, I would read this guide, the appropriate example, and the ipscend manual. I would set up an address using name services with IPNS.

Awesome IPFS lists some applications.

Peergos is a user-friendly file sync service based on IPFS that aspires to be a low-key social network. Maybe this is a good idea? See DIY socials.

The client software comes in desktop and Command-line flavours. Plus, there is also ipfs-companion: Browser extension that simplifies access to IPFS resources on the web which will work with either. go-ipfs will mount distributed IPFS resources as a local FS.

So, would I actually do this? I did for a bit. But I do not actually have any IPFS services that I use, so I was gaining nothing from it, while at the same time it was absolutely mincing my battery, taking hours off the battery life with the heavy CPU demands. Maybe I would try it if I had a desktop machine that I was not relying on battery for? It feels like a difficult value proposition here; it would be more compelling if there were more people using it, but it is hard to get that critical mass while it is mostly smashing your battery so you can LARP as a 733+ haX0Rz doing s3kr!t stuff.

3 cjdns

cjdns is the technical meshnet infrastructure underlying the flagship Hyperboria network. Perhaps the Project Meshnet intro is good?

Cjdns implements an encrypted IPv6 network using public-key cryptography for address allocation and a distributed hash table for routing. This provides near-zero-configuration networking and prevents many of the security and scalability issues that plague existing networks.

There are many shambolic features of this system, including important onboarding websites mentioned in the documentation vanishing from the internet. Is this system so very good that all its users are leaving the public web behind and do not need the rest of us? Or is it just badly administered? Also, the security model seems startling:

…Once your node is running, you’re now a newly minted IPv6 host. Your operating system may automatically reconfigure network services to use this new address. If this is not what you intend, you should check to see that you are not offering more services than you intended to. ;)

I was not aware ;) was the international symbol for putting blackhat bullseye on my laptop. So the default configuration is to set up your machine as a naked node on the uncensorable darknet. I’m pretty sure this project is not for humans, since you must now be a firewall expert too it seems, and enjoy securing IPv6 services.

Anyway, if I understand correctly, unlike IPFS which is an application protocol, this is a new network layer which leaves you serving the old app protocols via some lower-level p2p network protocol except to Macedonian teen hackers. I’m not saying this is useless, but the onboarding looks rough, unless you have Macedonian teen hacker friends.

3.1 Zeronet

Zeronet promises “Open, free and uncensorable websites, using Bitcoin cryptography and BitTorrent network”.

I am not convinced yet by this promise purely because there were too many buzzwords at once and I am grumpy today. Judge for yourself, though.

How does it work?

  • After starting zeronet.py you will be able to visit zeronet sites using http://127.0.0.1:43110/{zeronet_address} (e.g. http://127.0.0.1:43110/1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D).
  • When you visit a new zeronet site, it tries to find peers using the BitTorrent network so it can download the site files (html, css, js…) from them.
  • Each visited site is also served by you.
  • Every site contains a content.json file which holds all other files in a sha512 hash and a signature generated using the site’s private key.
  • If the site owner (who has the private key for the site address) modifies the site, then he/she signs the new content.json and publishes it to the peers. Afterwards, the peers verify the content.json integrity (using the signature), they download the modified files and publish the new content to other peers.

This BitTorrent thing sounds practically identical to the dat one although presumably with some protocol differences that us user schlubs don’t care about.

3.2 I2P

The invisible internet project

I2P is an anonymous network built on top of the internet. It allows users to create and access content and build online communities on a network that is both distributed and dynamic. It is intended to protect communication and resist monitoring by third parties such as ISPs.

It seems somewhat Tor-like but with less interest in proxying to the normal internet, rather funnelling you into local distributed services of some kind…?

3.3 Freenet

Freenet aims to be a peer data storage layer that you can build services on.

Freenet can be thought of as a large storage device. When you store a file in it, you receive a key which can be used to retrieve the file. When you supply Freenet with a key, it returns the appropriate file (if it is located). The storage space is distributed among all connected nodes on Freenet.

Freenet is a peer-to-peer network which is both decentralized and anonymized. The nodes that you connect to only know their nearest neighbours and have no idea about how the network as a whole is structured.

3.4 Secushare

Imagine Facebook, Whatsapp, Gmail and Skype rolled into one, without the centralized surveillance and control. Crazy? Well, it hasn’t been tried before, at least not our way. So let’s give it a try.

secushare employs GNUnet for end-to-end encryption and anonymizing mesh routing (because it has a more suitable architecture than Tor or I2P) and applies PSYC on top (because it performs better than XMPP, JSON or OStatus) to create a distributed social graph. Together, these technologies allow for distributed private social networking including more straightforward and secure e-mail, chat, exchange of content and a private web. It could even work out as a safer choice for the Internet of Things.

I’m not clear if they’ve shipped an actual product or not.

Figure 2

4 LBRY

LBRY:

LBRY differs from the status quo in three big ways:

  1. Coupled payment and access. If desired, the person who publishes to lbry://rhapsody-film#e1029aaa08bef8e9225efcbfb94a895e9bbdc8ea can charge a fee to users that view the content.
  2. Decentralised and distributed. Content published to LBRY is not specific to one computer or network, making LBRY robust to failure and disruption.
  3. Community controlled. No party other than the publisher (including us) can unilaterally remove or block content on the LBRY network.2

5 Incoming