When a p2p application starts the first time it needs to know how to contact the first peer. After that the first peer can give addresses of more peers. But how can the p2p application get the address of the first peer?
A solution would be a central server that maintains a list of peers. But this is not p2p. After all the point of p2p is that it doesn't need central servers.
However in the quest to understand how to boostrap p2p really independent of a central service, I feel somewhat cheated, because IPFS also would need to bootstrap. It's sort of kicking the can down the road. How does IPFS bootstrap?
Is there really a way to bootstrap p2p without reliance on a list of central servers?
I thought about a hardcoded algorithmic series of addresses. Let's say by using a repeatable random generator. An example: the first address is jec6r5bz.io, then 0ueasze6.ch, etc., I made up the addresses. The project would register the first few addresses and if they are blocked, register more addresses on that series. This way the application can bootstrap independent of a specific central server because it can try the next address if it doesn't work.
The problem is, as you've said, very generally and applies to almost all p2p applications. The Bitcoin client has a series of bootstrapping attempts (https://en.bitcoin.it/wiki/Network#Bootstrapping) which involves a list of hardcoded DNS and IP addresses, and establishing an IRC connection to find peers. I think using IPFS for bootstrapping is a pretty good idea IMO. At least you're relying on an already-decentralized system and not on a central server.
I've been thinking about finding simple solutions to the problem like treating the IP space like a binary search space or generally finding solutions that are better than random sampling (More methods are compared in https://link.springer.com/content/pdf/10.1007%2F978-3-642-01...)
I anyone can recommend more resources for learning about bootstrapping methods or has examples of how other apps solve the problem, please let me/us know!
> Is there really a way to bootstrap p2p without reliance on a list of central servers?
Yes. A list of nodes. The key part is that for p2p bootstrap you don't need to learn about any special node in the network, just any node will do. A set of initial, long-lived contacts could be embedded in a link when sharing content , it can be loaded from a file, it can be obtained via configurable DNS lookups. If your network were so niche that you couldn't even afford a server and all you have is an anonymous internet forum then you could even post a pastebin with some IPs and ports every now and then and prompt users to paste that.
And once you have bootstrapped yourself you can keep a local cache of long-lived nodes which you can also provide to others.
It seems philosophically hard, but practically speaking you only need to get in once and there are so many ways to do that. A well-known, server-like points of contact are mostly used because they are the most convenient and reliable option, but they're not really the only option.
So for automatic bootstrap use a hard-coded or generated list of long-lived nodes or a list downloaded from different p2p systems. The hard-coded list could be updated with a new version. Additionally allow manual input of addresses.
Or you defer bootstrap until you have some channel on which you can piggyback.
For example if you download some file then you need to exchange some content-identifying token anyway, a magnet link in bittorrent. That link can also contain a bunch of nodes the source peer knows, preferably the longest-lived ones it knows. Depending on where it's used you could also turn this into a QR-code, NFC peer or whatever.
Practically speaking an implementation will already contain a bootstrap list or might download it from somewhere else on first start. But that's just for convenience.
Bootstrapping P2P networks in a decentralized fashion is indeed a hard problem.
IPFS bootstraps by either having a static list of bootstrap peers (essentially normal nodes in the network) that are run by trusted organizations (currently some of them are run by Protocol Labs who also develop IPFS, libp2p and some other software), which is similar to how many P2P networks does bootstrapping. Or, if there is any nodes on the local network, those can also be discovered via mDNS that allows discovery on local networks. Once you've found one peer, you can exchange "address books" and find more peers.
Other ways, that I don't think are implemented anywhere in the wild as far as I know, is to do some sort of random-walk of the IP address space and just try to connect to random hosts, see if they are nodes. Obviously not super cheap, especially compared to bootstrap lists, and introduces some issues if the protocol is normally run on random ports and/or if NAT is being a particular bitch for the protocol to work around, which it tends to be. Coupled together with a local cache of previously seen nodes, it'll make initial bootstrap a bit slow but very resilient, and future disconnect/reconnects faster, but at least it'll be fully decentralized.
One could also imagine with the proliferation of new P2P technology like blockchains and eventually Matrix moving to P2P, that you might be able to use those for bootstrapping as well. If someone runs a Ethereum node with some Ether, maybe providing bootstrapping functionality for other networks could be something that could work via smart contracts? Just spit-balling here, but more out-of-band channels could be added for the discovery of peers.
Edit: a free-time curiosity of mine has always been to use inaudible sound for humans as a discovery protocol as well, but haven't really looked into that too much. If the nodes could use the microphone to pickup bytes from broadcasting nodes via audio, one might be able to put speakers in cities that aren't really noticeable for humans (but maybe too much for animals/other tech?) that allows nodes to bootstrap via it.
I work on a decentralised blockchain — the way we bootstrap new nodes onto the network is through a ‘topology’ file which contains peers they want to connect to on launch. This is literally just IP addresses and ports. The file could then be updated by those nodes sharing their topology files on new connections, but we don’t do that.
Sounds like you should just use libp2p (which is what IPFS is using for P2P networking) in whatever flavor you're building your blockchain with! At least you still have the same bootstrapping technology you're doing now, but with the added benefit of peers exchanging existing connections when new peers connects :)
I’ll check it out! Fwiw I work on Cardano but higher up the stack than networking, so I’m not 100% sure what happens there except for what I see as a ‘user’ of it.
When a p2p application starts the first time it needs to know how to contact the first peer. After that the first peer can give addresses of more peers. But how can the p2p application get the address of the first peer?
A solution would be a central server that maintains a list of peers. But this is not p2p. After all the point of p2p is that it doesn't need central servers.
https://github.com/dennis-tra/pcp#how-does-it-work uses IPFS to contact the first peer. OK.
However in the quest to understand how to boostrap p2p really independent of a central service, I feel somewhat cheated, because IPFS also would need to bootstrap. It's sort of kicking the can down the road. How does IPFS bootstrap?
Is there really a way to bootstrap p2p without reliance on a list of central servers?
I thought about a hardcoded algorithmic series of addresses. Let's say by using a repeatable random generator. An example: the first address is jec6r5bz.io, then 0ueasze6.ch, etc., I made up the addresses. The project would register the first few addresses and if they are blocked, register more addresses on that series. This way the application can bootstrap independent of a specific central server because it can try the next address if it doesn't work.