erlang-and-elixir-blockchain-tech-deep-dive

This is the fourth and final part of our ‘Making Sense of Blockchain’ blog post series, here we make the case for using Erlang, Elixir and the BEAM VM to power your blockchain project. You can read part 1 of this series on ‘6 Blockchain Principles’ here.

For more great content and news, join our FinTech mailing list for more great content and industry and events news, sign up here >>

Theme IV

Why companies use Erlang & Elixir for their blockchain solutions?

Building a robust and scalable blockchain presents a number of challenges that a research and development team typically needs to address. Often the ambitious goals to drive decentralised consensus and governance require unconventional approaches to achieve extra performance and reliability.

Improved Transactions per Second (TPS) is the most common challenge that blockchain related use cases expose. It is inherently difficult to produce a distributed peer-to-peer (P2P) network that can register transactions into a single data structure.

Guaranteeing consensus while delivering high TPS throughput among a vast number of nodes active on the network is even more challenging. Also, the fact that most public blockchains need to operate in a non-trusted mode requires adequate mechanisms for validation, which implies that contextual data needs to be available on demand. A blockchain should also be able to respond to anomalies such as network connectivity loss, node failure and malicious actors.

All of the above is further complicated by the continuous growth of the blockchain data structure itself, which becomes problematic in terms of storage.

It is clear that, unless we are prepared to invest vast amounts of resources, we would benefit from a high-level language and technology that allows us to quickly prototype and amend our code as we work our way towards an optimal solution to respond to our use case. It’s also ideal if such technology offers strong network and concurrent capabilities. A technology built with distribution in mind that offers a friendly paradigm for asynchronous computation, and which does not collapse under heavy load, but rather degrades gracefully and keeps delivering when traffic exceeds capacity.

The Erlang Beam VM (available also in the Elixir syntax flavour) undoubtedly scores high on the above list of desirable features.

“We needed to build our own protocol blockchain to power this decentralized network and the team chose Erlang as the language to build it. Key among the reasons for choosing Erlang were safety, composability, networking, and robustness.”

Dal Gemmell, Head of Product at Helium Inc.

Erlang & Elixir’s strengths for blockchain

Fast development

The Challenge: Blockchain technology is present in extremely competitive markets, such as fintech, and it is critical for organisations operating in them to be able to release new features in time to attract new customers and retain existing ones.

The Answer: Both Erlang and Elixir are functional languages, operating at a very high level of abstraction which is ideal for fast prototyping and development of new ideas. By using these languages on top of the Beam VM, developers dramatically increase the speed to market when compared to other lower level or object-oriented technologies.

Solutions developed in Erlang or Elixir also lead to a significantly smaller code base, which is easier to test and adapt to changes of direction. This is really helpful when you proceed to fast prototyping new solutions and when you discover that amendments and upgrades are necessary, which is very typical in blockchain R&D activity. 

Both languages offer support for unit testing in their standard library. This enables developers to adopt Test Driven approaches ensuring the quality is preserved when modules and libraries get refactored. The common test framework also provides support for distributed tests and can be integrated with Continuous Integration frameworks like Jenkins. Both the Erlang and Elixir shells let the programmer flesh out ideas really fast and access running nodes for inspection.

Introspection

The Challenge: To keep a competitive advantage in a fast-changing market, it is critical for organisations to promptly identify issues and opportunities by extracting relevant information about their running systems so that actions can be taken to upgrade them where appropriate.

The Answer: Erlang and Elixir allow connection to an already running system and a status check. This is an extremely useful debugging tool, both in the development and production environment. Statuses of processes can be checked, deadlocks in the live system can be analysed. There are various metrics and tools that can show overload, bottlenecks and other key performance indicators. Enhanced introspection tools such as Erlang Solutions’ Wombat OAM are also helping with the identification of scalability issues when you run performance tests.

Networking

The Challenge: Delivering a highly scalable and distributed P2P network is critical for blockchain enterprises. It is important to rely on stable battle-proven network libraries as reliable building blocks for exploiting use case specific innovative approaches.

The Answer: Erlang and Elixir come with strong and easy to manage network capabilities. There is a proven record of successful enterprises that rely on the BEAM VM’s networking strengths; including Adroll, WhatsApp, Riot Games, Klarna, Bet365 and Ericsson. Most of their use cases have strong analogies with the P2P networking that is required to deliver a distributed blockchain.

Combined with massive concurrency, the networking makes Erlang and Elixir ideal for server applications and means it can handle many clients. The binary and bitstring syntax makes parsing binary protocols particularly easy.

“Erlang has a strong networking story – there’s a claim that something like 60-80% of internet traffic goes through some Erlang code somewhere along the line.”

Dal Gemmell, Head of Product at Helium Inc.

Massively concurrent

The Challenge: There is a weakness afflicting Bitcoin and Ethereum where the computation of a block is competitive rather than collaborative. There is the opportunity to drive a collaborative concurrent approach: i.e. via sharding so that each actor can compute a portion of a block.

The Answer: The BEAM VM powering Erlang and Elixir provides lightweight processes for applications. These are lightweight so that hundreds of thousands of them can run simultaneously. These processes do not share memory, communication is done over asynchronous messages (unlike goroutines) so there’s no need to synchronise them. The BEAM VM implementation also makes use of all of the available CPUs and cores. This makes Erlang and Elixir ideal for workloads that involve a huge amount of concurrency and consist of mostly independent workflows. This feature is especially useful in addressing the coordinated distribution of portions of work to compute a Merkle Tree of transactions.

High availability and resilience

The Challenge: These are the requirements for every type of application and even more so for competitive and highly distributed blockchain networks. The communication and preservation of a state need to be as available as possible to avoid inconsistent states, network forks and disruptions experienced by the users.

The Answer: The fault tolerance properties mentioned in the previous paragraph combined with built-in distribution leads to high availability even in cases of hardware malfunction. Erlang and Elixir have the built-in mnesia database system with the ability to replicate data over a cluster of nodes so if one node goes down, the state of the system is not lost.

Erlang and Elixir provide the supervisor pattern to handle errors. Computing is done in multiple processes and if an error occurs and a process crashes, the supervisor is there to handle the situation, restart the computing or take some other measures. This pattern lets the actual code be cleaned as error handling can be implemented elsewhere. As processes are isolated, they do not share memory meaning errors are localised.

Built-in distribution

The Challenge: This is highly relevant for trusted or hybrid networks where a central network takes authoritative decisions on top of a broader P2P network. Using the “out of the box” Erlang distribution and proven consistency approaches such as RAFT can be a quick win towards a fast prototype of a blockchain solution.

The Answer: Erlang and Elixir provide out of the box tools to run a distributed system. The message handling functionalities of the system are transparent, so sending a message to a remote process is just as simple as sending it to a local process. There’s no need for convoluted IDLs, naming services or brokers.

Safety and security

The Challenge: Among the security features that both trusted and untrusted blockchain solutions strongly require, is the critical protection of access to the state memory thus reducing the exposure to a range of vulnerabilities.

The Answer: Erlang and Elixir, just like many high-level languages, do not manipulate memory directly so applications written in Erlang and Elixir are effectively immune to many vulnerabilities like buffer overflows and return-oriented programming. Exploiting these vulnerabilities would require direct access to the memory, but the BEAM VM hides the memory from the application code.

Concluding Thoughts

“Working with our R&D partners, Erlang Solutions, was a very rewarding experience. The POA Netstat Agent exceeded our expectations and the team really went the extra mile to deliver this! When it comes to Elixir development, we highly recommend Erlang Solutions.”

Igor Barinov, Tech Lead at POA Network

While many business leaders are still trying to figure out how to put the technology to work for maximum ROI, most agree on two things: firstly, blockchain unlocks vast value potential from optimised business operations and secondly, that it’s here to stay too. 

The technology’s potential is massive, working products and fulfilled promises are beginning to take shape. What this means for our partners is that talk about blockchain implementation is no longer merely food for theoretical debate, it is now with a view to imminent business application based on an ever growing library of proven use cases.

We strongly recommend that organisations do not sit on the side waiting for others to create effective business cases but ensure that they are building strong in-house knowledge so that they are ready to make effective decisions to embrace blockchain technology when the time is right.  Even without a completely solid business case to implement, organisations should, at the very least, keep an eye on developments in blockchain tech so that they can take advantage of opportunities when they present themselves. 

Business leaders must start planning how to best use this transformative technology to unleash trapped value in key operational processes. It’s clear – blockchain should be on every company’s agenda, regardless of industry.

In this blockchain blog post series we have presented an in-depth discussion about the technology on a variety of technical, theoretical and human themes. We hope you will be inspired by reading, reflecting and taking an active stand in the discussion. We are here to help you avoid a ‘do nothing’ mindset and encourage you to consult with us on how to scale your distributed systems.