6

Design and evaluation of IPFS | Proceedings of the ACM SIGCOMM 2022 Conference

 2 years ago
source link: https://dl.acm.org/doi/abs/10.1145/3544216.3544232
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

ABSTRACT

Recent years have witnessed growing consolidation of web operations. For example, the majority of web traffic now originates from a few organizations, and even micro-websites often choose to host on large pre-existing cloud infrastructures. In response to this, the "Decentralized Web" attempts to distribute ownership and operation of web services more evenly. This paper describes the design and implementation of the largest and most widely used Decentralized Web platform --- the InterPlanetary File System (IPFS) --- an open-source, content-addressable peer-to-peer network that provides distributed data storage and delivery. IPFS has millions of daily content retrievals and already underpins dozens of third-party applications. This paper evaluates the performance of IPFS by introducing a set of measurement methodologies that allow us to uncover the characteristics of peers in the IPFS network. We reveal presence in more than 2700 Autonomous Systems and 152 countries, the majority of which operate outside large central cloud providers like Amazon or Azure. We further evaluate IPFS performance, showing that both publication and retrieval delays are acceptable for a wide range of use cases. Finally, we share our datasets, experiences and lessons learned.

Supplemental Material

Available for Download
pdf

References

  1. 2017. Filecoin: A Decentralized Storage Network. Technical Report. Protocol Labs.
  2. 2020. IPFS for Nextcloud. https://apps.nextcloud.com/apps/files_external_ipfs
  3. 2021. DNSLink Standard. https://www.dnslink.io/
  4. 2021. Multiformats - Self-describing values for Future-proofing. https://multiformats.io/
  5. 2022. IPFS Ecosystem directory. https://ecosystem.ipfs.io/.
  6. 2022. Udger Data v3 - 20220606-01. Retrieved 02 June 2022 from https://udger.com/
  7. Omar Abdullah Lajam and Tarek Ahmed Helmy. 2021. Performance Evaluation of IPFS in Private Networks. In 2021 4th International Conference on Data Storage and Data Engineering (Barcelona, Spain) (DSDE '21). Association for Computing Machinery, New York, NY, USA, 77--84.
  8. C Bommelaer de Leusse and Carl Gahnberg. 2019. The Global Internet Report: Consolidation in the Internet Economy. Internet Society (2019).
  9. Timm Böttger, Gianni Antichi, Eder L Fernandes, Roberto di Lallo, Marc Bruyere, Steve Uhlig, and Ignacio Castro. 2018. The elusive internet flattening: 10 years of IXP growth. RIPE 78 (2018).
  10. Antonio Carzaniga, Matthew J Rutherford, and Alexander L Wolf. 2004. A routing scheme for content-based networking. In IEEE INFOCOM 2004, Vol. 2. IEEE, 918--928.
  11. Ignacio Castro, Rade Stanojevic, and Sergey Gorinsky. 2013. Using Tuangou to reduce IP transit costs. IEEE/ACM Transactions on Networking 22, 5 (2013), 1415--1428.
  12. Wei Koong Chai, Ning Wang, Ioannis Psaras, George Pavlou, Chaojiong Wang, Gerardo Garcia de Blas, Francisco Javier Ramon-Salguero, Lei Liang, Spiros Spirou, Andrzej Beben, and Eleftheria Hadjioannou. 2011. Curling: Content-ubiquitous resolution and delivery infrastructure for next-generation services. IEEE Communications Magazine 49, 3 (2011), 112--120.
  13. Ian Clarke, Oskar Sandberg, Brandon Wiley, and Theodore W Hong. 2001. Freenet: A distributed anonymous information storage and retrieval system. In Designing privacy enhancing technologies. Springer.
  14. Kelly Clay. 2013. Amazon.com Goes Down, Loses $66,240 Per Minute. https://web.archive.org/web/20210307232341/https://www.forbes.com/sites/kellyclay/2013/08/19/amazon-com-goes-down-loses-66240-per-minute/
  15. Bram Cohen. 2003. Incentives build robustness in BitTorrent. In Workshop on Economics of Peer-to-Peer systems, Vol. 6. Berkeley, CA, USA, 68--72.
  16. Bram Cohen. 2008. The BitTorrent Protocol Specification v2. Retrieved 18 May 2022 from https://www.bittorrent.org/beps/bep_0052.html
  17. Devin Coldewey. 2020. Cloudflare DNS goes down, taking a large piece of the internet with it. Retrieved 18 May 2022 from https://techcrunch.com/2020/07/17/cloudflare-dns-goes-down-taking-a-large-piece-of-the-internet-with-it/
  18. Scott A Crosby and Dan S Wallach. 2007. An analysis of bittorrent's two kademlia-based dhts. Technical Report. Rice Technical Report.
  19. Christian Dannewitz, Dirk Kutscher, BöRje Ohlman, Stephen Farrell, Bengt Ahlgren, and Holger Karl. 2013. Network of Information (NetInf) - An Information-Centric Networking Architecture. Comput. Commun. 36, 7 (apr 2013), 721--735.
  20. Alfonso de la Rocha, David Dias, and Yiannis Psaras. 2021. Accelerating Content Routing with Bitswap: A Multi-Path File Transfer Protocol in IPFS and Filecoin. (2021).
  21. David Dias, Jeromy Johnson, and Juan Benet. 2020. Bitswap - Protocol Specification. Retrieved 01 June 2022 from https://github.com/ipfs/specs/blob/master/BITSWAP.md
  22. Jarret Falkner, Michael Piatek, John P. John, Arvind Krishnamurthy, and Thomas Anderson. 2007. Profiling a Million User Dht. In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement (San Diego, California, USA) (IMC '07). Association for Computing Machinery, New York, NY, USA, 129--134.
  23. Rodérick Fanou, Gareth Tyson, Pierre Francois, and Arjuna Sathiaseelan. 2016. Pushing the frontier: Exploring the african web ecosystem. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 435--445.
  24. Gnutella. 2009. Gnutella Protocol Specification. Retrieved 18 May 2022 from https://web.archive.org/web/20090331221153/http://wiki.limewire.org/index.php?title=GDF
  25. Kalman Graffi, Christian Gross, Dominik Stingl, Daniel Hartung, Aleksandra Kovacevic, and Ralf Steinmetz. 2011. LifeSocial. KOM: A secure and P2P-based solution for online social networks. In CCNC.
  26. Barbara Guidi, Marco Conti, Andrea Passarella, and Laura Ricci. 2018. Managing social contents in Decentralized Online Social Networks: A survey. Online Social Networks and Media 7 (2018).
  27. Anaobi Ishaku Hassan, Aravindh Raman, Ignacio Castro, Haris Bin Zia, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. 2021. Exploring content moderation in the decentralised web: The pleroma case. In Proceedings of the 17th International Conference on emerging Networking EXperiments and Technologies. 328--335.
  28. Sebastian A. Henningsen, Martin Florian, Sebastian Rust, and Björn Scheuermann. 2020. Mapping the Interplanetary Filesystem. 2020 IFIP Networking Conference (Networking) (2020), 289--297.
  29. Ralph Holz, Jens Hiller, Johanna Amann, Abbas Razaghpanah, Thomas Jost, Narseo Vallina-Rodriguez, and Oliver Hohlfeld. 2020. Tracking the deployment of TLS 1.3 on the Web: A story of experimentation and centralization. ACM SIGCOMM Computer Communication Review 50, 3 (2020), 3--15.
  30. Tomas Isdal, Michael Piatek, Arvind Krishnamurthy, and Thomas Anderson. 2010. Privacy-Preserving P2P Data Sharing with OneSwarm. In Proceedings of the ACM SIGCOMM 2010 Conference (New Delhi, India) (SIGCOMM '10). Association for Computing Machinery, New York, NY, USA, 111--122.
  31. Van Jacobson, Diana K. Smetters, James D. Thornton, Michael F. Plass, Nicholas H. Briggs, and Rebecca L. Braynard. 2009. Networking Named Content. In Proceedings of the 5th International Conference on Emerging Networking Experiments and Technologies (Rome, Italy) (CoNEXT '09). Association for Computing Machinery, New York, NY, USA, 1--12.
  32. M Frans Kaashoek and David R Karger. 2003. Koorde: A simple degree-optimal distributed hash table. In International Workshop on Peer-to-Peer Systems. Springer, 98--107.
  33. Sebastian Kaune, Konstantin Pussep, Christof Leng, Aleksandra Kovacevic, Gareth Tyson, and Ralf Steinmetz. 2009. Modelling the internet delay space based on geographical locations. In 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing. IEEE, 301--310.
  34. Prashant Khare, Mladen Karan, Stephen McQuistin, Colin Perkins, Gareth Tyson, Matthew Purver, Patrick Healey, and Ignacio Castro. 2022. The Web We Weave: Untangling the Social Graph of the IETF. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 16. 500--511.
  35. David Koll, Dieter Lechler, and Xiaoming Fu. 2017. SocialGate: Managing large-scale social data on home gateways. In IEEE ICNP.
  36. Protocol Labs. 2021. Direct Connection Upgrade through Relay. Retrieved 01 June 2022 from https://github.com/libp2p/specs/blob/master/relay/DCUtR.md
  37. Protocol Labs. 2021. Merkle Directed Acyclic Graphs (DAGs). Retrieved 18 May 2022 from https://docs.ipfs.io/concepts/merkle-dag/
  38. Protocol Labs. 2022. AutoNAT. Retrieved 01 June 2022 from https://github.com/libp2p/specs/blob/master/autonat/README.md
  39. Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2019. Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation. In Proceedings of the 26th Annual Network and Distributed System Security Symposium (NDSS 2019).
  40. Guoli Li, Vinod Muthusamy, and Hans-Arno Jacobsen. 2008. Adaptive content-based routing in general overlay topologies. In ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing. Springer, 1--21.
  41. Thomas Mager, Ernst Biersack, and Pietro Michiardi. 2012. A measurement study of the Wuala on-line storage service. In 2012 IEEE 12th International Conference on Peer-to-Peer Computing (P2P). IEEE.
  42. Essam Mansour, Andrei Vlad Sambra, Sandro Hawke, Maged Zereba, Sarven Capadisli, Abdurrahman Ghanem, Ashraf Aboulnaga, and Tim Berners-Lee. 2016. A demonstration of the solid platform for social web applications. In WWW.
  43. Eva Mathews. 2021. Amazon cloud outage hits major websites, streaming apps. Retrieved 18 May 2022 from https://www.reuters.com/article/amazon-com-outages-idCAKBN2IM1U0
  44. Petar Maymounkov and David Mazieres. 2002. Kademlia: A peer-to-peer information system based on the xor metric. In International Workshop on Peer-to-Peer Systems. Springer, 53--65.
  45. Stephen McQuistin, Mladen Karan, Prashant Khare, Colin Perkins, Gareth Tyson, Matthew Purver, Patrick Healey, Waleed Iqbal, Junaid Qadir, and Ignacio Castro. 2021. Characterising the IETF through the lens of RFC deployment. In Proceedings of the 21st ACM Internet Measurement Conference. 137--149.
  46. Fabrizio Parrillo and Christian Tschudin. 2021. Solid over the Interplanetary File System. In 2021 IFIP Networking Conference (IFIP Networking).
  47. Charith Perera, Susan YL Wakenshaw, Tim Baarslag, Hamed Haddadi, Arosha K Bandara, Richard Mortier, Andy Crabtree, Irene CL Ng, Derek McAuley, and Jon Crowcroft. 2017. Valorising the IoT databox: creating value for everyone. Transactions on Emerging Telecommunications Technologies 28, 1 (2017).
  48. Michael Piatek, Tomas Isdal, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. 2007. Do Incentives Build Robustness in BitTorrent?. In 4th USENIX Symposium on Networked Systems Design & Implementation (NSDI 07). USENIX Association, Cambridge, MA. https://www.usenix.org/conference/nsdi-07/do-incentives-build-robustness-bittorrent
  49. Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. 2019. Challenges in the decentralised web: The mastodon case. In Proceedings of the Internet Measurement Conference. 217--229.
  50. CAIDA AS Rank. 2022. urlhttp://as-rank.caida.org/.
  51. Sylvia Ratnasamy, Paul Francis, Mark Handley, Richard Karp, and Scott Shenker. 2001. A Scalable Content-Addressable Network. SIGCOMM Comput. Commun. Rev. 31, 4 (aug 2001), 161--172.
  52. Drew Roselli, Jacob R Lorch, and Thomas E Anderson. [n.d.]. A Comparison of File System Workloads. ([n. d.]), 14.
  53. Mathieu Rosemain and Raphael Satter. 2021. Millions of websites offline after fire at French cloud services firm. Retrieved 18 May 2022 from https://www.reuters.com/article/us-france-ovh-fire-idUSKBN2B20NU
  54. Antony Rowstron and Peter Druschel. 2001. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In IFIP/ACM International Conference on Distributed Systems Platforms and Open Distributed Processing. Springer, 329--350.
  55. Osama Saleh and Mohamed Hefeeda. 2006. Modeling and Caching of Peer-to-Peer Traffic. In Proceedings of the 2006 IEEE International Conference on Network Protocols. IEEE.
  56. Vasco Santos and Steven Allen. 2021. IPNS - Inter-Planetary Naming System. https://github.com/ipfs/specs/blob/bab189ee61c316eb3d371c7270abef97641e7ed9/IPNS.md
  57. Stefan Saroiu, P Krishna Gummadi, and Steven D Gribble. [n.d.]. A Measurement Study of Peer-to-Peer File Sharing Systems. In Multimedia Computing and Networking 2002 (2001-12-10), Vol. 4673. SPIE, 156--170.
  58. Lorenz Schwittmann, Christopher Boelmann, Matthaus Wander, and Torben Weis. 2013. SoNet-Privacy and Replication in Federated Online Social Networks. In Distributed Computing Systems Workshops.
  59. Ion Stoica, Robert Morris, David Karger, M Frans Kaashoek, and Hari Balakrishnan. 2001. Chord: A scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM Computer Communication Review 31, 4 (2001), 149--160.
  60. Daniel Stutzbach and Reza Rejaie. 2006. Improving lookup performance over a widely-deployed DHT. In Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications. IEEE, 1--12.
  61. Daniel Stutzbach and Reza Rejaie. 2006. Understanding Churn in Peer-to-Peer Networks. In Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement (Rio de Janeiro, Brazil) (IMC '06). Association for Computing Machinery, New York, NY, USA, 189--202.
  62. Sanaz Taheri-Boshrooyeh, Alptekin Küpçü, and Öznur Özkasap. 2015. Security and privacy of distributed online social networks. In Distributed Computing Systems Workshops.
  63. Dominic Tarr, Erick Lavoie, Aljoscha Meyer, and Christian Tschudin. 2019. Secure Scuttlebutt: An Identity-Centric Protocol for Subjective and Decentralized Applications. In Proceedings of the 6th ACM Conference on Information-Centric Networking (Macao, China) (ICN '19). Association for Computing Machinery, New York, NY, USA, 1--11.
  64. Dennis Trautwein. 2021. Nebula - A crawler for networks based on the libp2p DHT implementation. https://github.com/dennis-tra/nebula-crawler
  65. David Vorick and Luke Champine. 2014. Sia: Simple Decentralized Storage. Technical Report. Nebulous Inc.
  66. Liang Wang and J. Kangasharju. 2013. Measuring large-scale distributed systems: case of BitTorrent Mainline DHT. In Peer-to-Peer Computing (P2P), 2013 IEEE Thirteenth International Conference on. 1--10.
  67. Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu. 2005. Finding Collisions in the Full SHA-1. In Advances in Cryptology - CRYPTO 2005. Springer Berlin Heidelberg, 17--36.
  68. Web Storage Index. 2022. https://www.storageindex.io/.
  69. Sam Williams, Viktor Diordiiev, Lev Berman, India Raybould, and Ivan Uemlianin. [n.d.]. Arweave: A Protocol for Economically Sustainable Information Permanence. Technical Report. arweave.org.
  70. Scott Wolchok and J. Alex Halderman. 2010. Crawling BitTorrent DHTs for Fun and Profit. In 4th USENIX Workshop on Offensive Technologies (WOOT 10). USENIX Association, Washington, DC. https://www.usenix.org/conference/woot10/crawling-bittorrent-dhts-fun-and-profit
  71. Lixia Zhang, Alexander Afanasyev, Jeffrey Burke, Van Jacobson, kc claffy, Patrick Crowley, Christos Papadopoulos, Lan Wang, and Beichuan Zhang. 2014. Named Data Networking. SIGCOMM Comput. Commun. Rev. 44, 3 (jul 2014), 66--73.
  72. Ben Y Zhao, Ling Huang, Jeremy Stribling, Sean C Rhea, Anthony D Joseph, and John D Kubiatowicz. 2004. Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications 22, 1 (2004), 41--53.

Comments

0 Comments


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK