Yesterday I did a lot reading Kademlia and DHT in general. I didn't really have any insights so I didn't write anything yesterday. Today I did some more reading on decentralized networks here: https://docs.google.com/presentation/d/11qGZlPWu6vEAhA7p3qsQaQtWH7KofEC9dMeBFZ1gYeA/edit#slide=id.g1718cc2bc_08645 I only skimmed through the last part. It was a nice presentation but it didn't cover anything on adversial nodes, so it doesn't apply 100% to Captain Dirgo. It did talk about gossip protocol which seemed interesting but I didn't delve much into it. I also went through and analyzed the code of the most popular javascript implement on Mainline DHT (the one that bittorrent uses), so I could get a further handle on what is involved in a DHT network. (notes below) My plan now is to either modify the DHT system itself or rewrite it to make it work with browser nodes using WebRTC and CPU PoW WebWorkers to handle the karma part (where each user gets a vanity score for all their upvotes on their comments, and well as flair, etc.) I'll think more about this tomorrow. I was also considering a coin. I do worry that anytime real money is involved, the incentives get skewed. My main concern is that rather than use PoW to support the network, and fight spam, people would just use PoW to generate coins for themselves. Since custom programs created to on dedicated hardware would produce much more hashpower than anything I could write in javascript or wasm, it would mean that 99.99% of coins would go to these people and anyone trying to save comments attached to urls would basically get nothing in comparison for their work. In other words, the worthlessness of just vanity karma that can't be traded keeps the network more stable then if it was actually worth something. That said, I would like a global state or something to point all that PoW work at. Right now I don't have anything to point the work towards (it doesn't need to be pointed at anything, but might as well not waste it). I think maybe I could use it to have a canonical state for all url mesages, and a user state in general. So maybe that will work. Yesterday I did a lot reading Kademlia and DHT in general. I didn't really have any insights so I didn't write anything yesterday. Today I did some more reading on decentralized networks here: https://docs.google.com/presentation/d/11qGZlPWu6vEAhA7p3qsQaQtWH7KofEC9dMeBFZ1gYeA/edit#slide=id.g1718cc2bc_08645 I only skimmed through the last part. It was a nice presentation but it didn't cover anything on adversial nodes, so it doesn't apply 100% to Captain Dirgo. It did talk about gossip protocol which seemed interesting but I didn't delve much into it. I also went through and analyzed the code of the most popular javascript implement on Mainline DHT (the one that bittorrent uses), so I could get a further handle on what is involved in a DHT network. (notes below) My plan now is to either modify the DHT system itself or rewrite it to make it work with browser nodes using WebRTC and CPU PoW WebWorkers to handle the karma part (where each user gets a vanity score for all their upvotes on their comments, and well as flair, etc.) I'll think more about this tomorrow. I was also considering a coin. I do worry that anytime real money is involved, the incentives get skewed. My main concern is that rather than use PoW to support the network, and fight spam, people would just use PoW to generate coins for themselves. Since custom programs created to on dedicated hardware would produce much more hashpower than anything I could write in javascript or wasm, it would mean that 99.99% of coins would go to these people and anyone trying to save comments attached to urls would basically get nothing in comparison for their work. In other words, the worthlessness of just vanity karma that can't be traded keeps the network more stable then if it was actually worth something. That said, I would like a global state or something to point all that PoW work at. Right now I don't have anything to point the work towards (it doesn't need to be pointed at anything, but might as well not waste it). I think maybe I could use it to have a canonical state for all url mesages, and a user state in general. So maybe that will work. -- bittorrent-dht: //start it up and listen const dht = new DHT() dht.listen(20000, function () { console.log('now listening') }) //this will state that it found a peer serving the particular infoHash (this can be thought // of part of the value of the key) dht.on('peer', function (peer, infoHash, from) { console.log('found potential peer ' + peer.host + ':' + peer.port + ' through ' + from.address + ':' + from.port) }) // find peers for the given torrent info hash. peers get sent as 'peer' events, listened to as above dht.lookup(parsed.infoHash, <callback when terminated>) If we take a closer look: lookup This calls _closest _closest creates a new table (k-bucket) for the infohash calls k-rpc.closest whenever it gets a reply it stores it in the table as a cache (note this is not in k-rpc so it doesn't mess with k-rpc tables) the peers themselves are the values (since this is bittorrent, after all) ??? do we follow bittorrent here, or do we directly return a result? announce(infoHash, [port], [callback]) If there isn't a table for the torrent already, calls lookup, and callsback announce after lookup is done. Then calls k-rpc.queryAll() with the q field set to 'announce_peer' and the a field with the port (i guess the address is taken from the message) The values of the queryAll are from the k-bucket table according to the infohash. put(opts, callback) // writes arbritrary payload Has a bunch of logic for multiple key signed data But otherwise just runs queryAll with the message 'put' get // reads arbritrary payload calls _closest calls k-rpc.closest receiving (in bittorrent-dht) onquery: //receives a message from a peer possible messages: ping: just reply with our id find_node: calls k-rpc.nodes.closest(). k-rpc.nodes is k-bucket table, so it pull out all of the closest nodes from its cache get_peers: if it has a list of peers for the infohash, will return it, otherwise calls nodes.closest for the infohash to get the closest nodes it knows announce_peer: validates token against calculated host token. then adds the peer (no matter how close it is to the infohash) get: if it's got the value cached, it gives the value, otherwise it calls nodes.closest() agai, like above put: puts the token, does special stuff if mutable (that I don't want to look at now) then responds. onnode: //called whenever a node is added by internal code, re-emits it to client k-rpc closest calls k-rpc._closest Makes an empty k-bucket table calls its subfunction kick() with an empty k-bucket table kick will use "nodes" which is the user's own node's k-bucket table if not there, it'll use bootstrap to query nodes kick will then query all the nodes in the k-bucket using the k-rpc-socket.query that it can and query any it didn't query before (using the 'queried' boolean array to keep track) with a callback of the function, afterQuery afterQuery gets the response back, if it's bad it calls kick() again if its good, it'll add the peer it tried to connect to to its table and call the "visit" callback argument if it was passed in then call kick again too