https://flexathon.net/twitter/index.html
What is this?
A demonstration of a Twitter clone operating as a decentralized application.
Decentralized applications (or DAPPS) will form the corner stone of #web3 technologies, allowing users to completely control their data, accessibility to it and montetization.
Why choose Twitter?
A Twitter clone is a great demonstrator for this technology as everyone is familiar with it or at least social networks in general.
This common knowledge provides an opportunity to create an interactive demonstration that requires little learning curve to use.
From a technical point of view Twitter is quite a complicated platform, a clone of it implemented with #web3 technologies allows us to push and find the limits. Twitter also has a huge amount of data which we have managed to acquire allowing us to test at scale.
We have over 100TB (about 20,000 DVDs) of tweets and image data, all of which will be available to this Twitter DAPP.
What can I do in the demo?
You will be able to perform all the basic actions you can on the real Twitter such as:
- Register an new account
- Make tweets
- Reply to tweets
- Retweets and likes
- Search users
- Search hashtags
- Follow others
Because we are also importing real tweets and users from the data we have, it is likely that if you have a Twitter account it will already be there. You can gain access instructions to your imported Twitter account by tweeting / messaging @fuserleer.
The demonstration is currently importing tweets and users beginning 1st January 2021. If you follow a popular account in the demonstration you will see historic tweets made by them appear as they are imported (which is pretty cool!).
Later more advanced demonstrations will import starting ~2012, have more features and ingest the full 100TB+ data set.
Don't panic! We only have your public user and tweet information!
Not passwords, not email addresses, not private messages!
What is missing in the demo?
Lots, as the currently implemented features are driven by the research needs. We plan to add more features over the coming weeks, the main ones being
- Upload/change profile/banner images
- Display embedded media content
- Tipping functionality
- Direct messages
- Trending hashtags
As our data feed is not a 100% snapshot of Twitter, some tweets, retweets, replies and profiles will not be imported. By the same account, many profile images and banners are missing from our media data set with the default profile image and banner image being used in this cases.
What this demo is not?
This demonstration is just that, a demonstration, which we thought would be cool to share in the hope of triggering some imaginations of what might be possible in the future.
It is not a complete, polished product. There will be bugs, things that don't work or break. The website was built by one person over the course of a few weeks who is not a competent web-dev by any measure of a yard-stick.
How does it work?
This #web3 Twitter DAPP is completely hosted within a decentralized network called Cassandra, which you can think of as an internet within the internet. All of the data, tweets, images and even the website itself live within this decentralized network. Not only that, multiple large scale DAPPS could be build and deployed such as marketplaces, forums, websites, etc.
Cassandra is an open-source distributed ledger technology (similar to Bitcoin or Ethereum) network we use at Radix DLT to research new, cool and exciting things we can include on our product roadmap. It represents a number of novel research achievements and theory unification not seen any other technology.
This research is notable because while Twitter has been built on other platform such as Ethereum, it could not operate at the scale of data or throughput seen in this demonstration. Any attempts to replicate this demonstration like for like on other DLT platforms would require various dependencies on other independent technologies, increasing complexity and potentially decreasing security or reliability.
For example, to do the same in Ethereum would require each node in the network to have a specification of at least 64GB RAM and 100TB of storage. For comparison each node in the Cassandra network can operate with just 8GB RAM and 1TB of storage thanks to a technique called Sharding.
For the more technical, Cassandra is a novel hybrid consensus mechanism which provides high levels of safety, liveness and throughput. This demonstration is operating on a total of 64 replicas, split into 8 sets of 8 replicas which are geo-located in 4 continents. Each group is responsible for a large set of shards (2253) and each tweet touches multiple shards and replica sets. Commits are atomic across shards and must satisfy an 'all or none' policy. All events undergo full consensus, signing, verification and execution as would be in a deployed network.
Operating conditions are also sub-optimal as the feed data we have is not complete, therefore a reply or retweet may reference a parent tweet which is not present in the feed data. This is not discovered until the action is executed after undergoing the first round of consensus (there are two rounds). Consensus still must complete with replicas agreeing the tweet should fail. Around 10-15% of tweets presented from the feed fail due to this issue which is a nice side-effect to test many failing conditions.
While the total quantity of replicas is low in this deployment, each tweet touches on average 4 sets, giving a total quorum size involved in consensus of at least 32 replicas. From a focussed research perspective a greater quantity of replicas presently does not offer much additional value for the associated logistical cost. Later deployments and demonstrations will have greater quantity of both replicas and replica sets as required and we will also leverage the generous community to increase further at testing progresses.
Cassandra is still in heavy research and is completely unoptimized. When interacting with the demonstration actions to the ledger require 10-15 seconds to become final with most of that latency due to all actions being cross shard and strictly atomic. There are optimizations possible which should reduce finality latency by 50% or so. Demonstration throughput is held constant at ~500 tweets/users/images per second.
No comments:
Post a Comment