Peppermint tea totally doesn't work, does it? been there, done that. Have you tried Tic-Tacs? they work for my IBS (but rot my teeth, swings and roundabouts.)
Anyway... this is all very P2Pish -- I'm watching with interest...
What about a sort of middle way kludge, if you had a trusted 3rd party operating with partitioned big fat servers which could happily hum away live all the time?
Or is that too obvious a question which must have been discounted in the first 2 minutes?
P2P is one form of truly distributed systems, there are many others and all of them fall because of unreliability.
The web of today is much more organic (completely decentralized) that it was years ago and with broadband and 3G coming to more and more countries it'll be more organic every day.
Unfortunately, organic networks are too slow (not only because of personal links but because the data is *everywhere* in *any* format) and navigating through such network requires both background process and artificial intelligence (allied with local caches, classification algorithms, etc) but none of that is really available nowadays...
OK... so if I'm following you rightly...
Bandwidth will cease to be a problem at some point, because broadband is going to be pervasive and cheap and always on.
Hard drives, i.e. storage, is going to cease to be a problem because it's going to become a lot more robust (droppable), and far cheaper for dizzyingly huge amounts of room to put everything. Unlike my flat.
So if computers are always on, and bandwidth is always on, then the edges become more robust themselves because everything you need there is always on.
Aaah, but what about mobile devices? They're not always on, nor are they always connected. Well... I think that's a problem that's going to solve itself too. We're seeing an increase in small-form, high-storage, fast computers like the Mac Mini, which people are using for their media, and to back up their laptops.
As the "back your shit up else you lose it" message gets hammered home (in the UK, through ads from BT selling a back-up service), more people are going to replicate their data on an always-on machine that has way more storage than their laptop has.
The logical conclusion to this is that people will start to want access to their big fat storage drive at home whilst they are away, and I think we'll see the development of server-like software that lets people get at their stuff without having to actually turn their Mac Mini into a proper server.
And as video and TV-on-demand style services becomes more important to consumers, so ISPs are going to have to make their connections more robust so that you don't end up getting some horrible, jittery playback when watching your favourite TV.
So yeah, I reckon it'll solve itself.
Oh, and try ginger cordial instead of peppermint tea.
IMHO you are on the right track here. The technology to enable this is already inside the data centers.
There's a huge shift from making apps depend on heavy interactions such as SQL and IMAP to making stuff just run on key/value pairs. Look at memcache, or all the "anything over Freenet" stuff that could still be dug up and adapted to new DHTs.
Once you get to key/value instead of structured query, you can decentralize radically -- see Git, then putting a wiki on Git, and then putting anything on a wiki.
Think about a Bluetooth/SATA box that you put a cheap commodity hard drive in, and that does a slo-mo DHT with everyone else's Bluetooth/SATA box.
Justin: actually, it's been about the only thing that has worked -- well, enteric peppermint oil capsules, which are futurist high-tech cups of tea in pill form.
Cait: no, that absolutely makes a lot of sense, but it somewhat collapses into "Why don't we just run everything on centralised servers anyway?" Which may be the real answer, but one that I want to avoid while I'm thinking out the ramifications of doing the very opposite.
Renato + Cait: Yes, P2P isn't a good answer for the network itself, so you need semi-centralised fallbacks (most P2P networks end up with some kind of centralised directory service, even the Net as a whole). One meme that quickly emerged was the "thin server", which is a caching proxy somewhere that holds only the thinnest copy of transactions, and syncs with your private edge server as often as it can.
Don: Yes! This is pretty much the scenario that I'm hearing smart folk regularly stumble onto (and I do think software revelations like git and other distributed version control systems inspire a lot of this thinking). And of course, right on time...
I think IPv6, if it ever gets inevitably deployed, solves many of the "reliability" and all of the NAT problems. I've actually noticed something interesting recently on bittorrent -- Azureus at least seems to do IPv6 now, and when you're connected to a torrent there's lots of windows boxes out there which are auto-IPv6ing themselves using MS's Miredo thingie. Which is really interesting, because of course nobody's firewalls today filter IPv6 properly...
I should be talking about this when less tired, but I'm partial to Don's point -- it's about the identifiers, and it's about the diffs.
I'm personally fascinated by the distinction between two types of 'personal' data: commodity data (i.e. files that could be retrieved from a multiplicity of sources) that you've paid money for; and unique individual data that you've created and keep online. There are varieties of commodity data, and it's an odd spectrum from, say, the base files in your OS through varieties of digital media encodes to your document and photo archive.
In the past decade, as disk sizes have grown exponentially, I'm pretty certain that the proportion of commodity data has risen on a higher curve than personal data to fill much of that space.
(Abstracting, though, is something like your iTunes library personal data? It makes me wonder whether the best definition of non-commodity data is 'data that marketers could make money from if they had access to it.')
In strictly digital terms, even your personal photos are, from one perspective, just diffs. But there's a cultural wrapping. If you lose commodity data, you just want it back without paying for it. If you lose personal data, expense is often not an issue.
Ultimately, I think the edge scenario is more compelling, as long as there are programmatic structures in place to be rid of permanent deletion forever. (Or, to put it another way, to have cloaked deletion but ubiquitous undeletion.)
And that, of course, raises privacy issues as well.
Commenting by HaloScan
