Formulate – Build forms, surveys, and quizzes entirely in code
source link: https://formulate.dev/blog/introducing-formulate
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Formulate is a new type of form-builder. Unlike most other tools in this space, Formulate lets you build forms using code, which is a great way to express conditional logic and other forms of dynamic behavior.
You can use it to build the kinds of things you’d expect: forms, surveys, and quizzes. But it also works for basic apps, games, or even as an intro to coding.
Try it right away, look at the docs or some examples, or keep reading for more context about what this tool is and the problems it’s trying to solve.
No-code tools are more popular than ever before. You’ll hear about how they’re democratizing programming so everyone can write software, and this is true … to some extent. There are a ton of legitimate uses for tools that allow non-coders to build software.
The unfortunate thing is, though: we’ve ended up with a largely bimodal distribution of available options: either you write software from scratch yourself, or you use a visual builder of some sort, typically a drag-and-drop GUI.
This should really look more like a spectrum: programmers could benefit from some of the advantages that no-code tools provide without having to jump all the way to a clicky GUI. This works the other way, too: non-coders are perfectly capable of programming their tools in small ways without having to write an entire full-stack webapp from scratch. Ask any Excel (or AppleScript) power user!
What’s the big deal with code, though? How is coding business logic any more powerful than defining logic in a visual way? Here’s a quick run-down:
Programmability
What does it even mean?
At its core, programmability is a means for end-users to have a system behave exactly they way they want it to, without having to make feature requests to the developers.
It’s literally impossible for a non-programmable system to anticipate everything a user may want to do with it, so the designers have to make assumptions. This allows for a wider audience (you don’t need to know how to code to use this thing) at the cost of power and expressiveness.
A programmable tool makes the opposite tradeoff. The audience is necessarily narrower, but you can use the tool in all sorts of interesting ways that the designers may not even have imagined.
While those are the extremes, it’s possible for tools combine the two, offering a programmable substrate that some users build plugins on, or simply as a secondary interface.
Programmable Systems
HyperCard
HyperCard was a tool to build applications interactively on early Macintosh OSes. You could build visually, and drop down into code when you needed to.
HyperCard used the HyperTalk programming language, which attempted to use natural language to construct programs. HyperTalk eventually led to the creation of AppleScript.
Lisp is one of oldest programming languages, second only to Fortran.
Many ideas in CS that are common today are from Lisp: trees, dynamic types, conditionals, recursions, and REPLs.
Lisp (and its dialects) are programmable programming languages. That’s a bit of a mouthful, so let’s break it down before we move on. Feel free to skip this mini-tutorial if you already know what Lisp macros are.
What is a programmable programming language?
Take a "regular" programming language like Javascript. You define a function like so:
You can then call the function with numbers as arguments:
You can also call the function with more complex arguments:
Importantly, the add
function has no knowledge of the fact that you performed all those multiplications to compute the first argument, and it certainly doesn’t know that you called Math.pow
to compute the second.
The Javascript runtime evaluates these two arguments before invoking add
, so that by the time add
is called, the call looks something like this:
In a language like Javascript, you can’t write this function in a way that allows add
to inspect the operations that went into calculating its arguments.
In a Lisp dialect, though? It’s easy! You can write a special kind of function called a macro that receives all it’s arguments as-is. The macro can inspect the whole thing before deciding what to do, making some truly dynamic behavior possible.
Here’s the same add
function in Clojure, a Lisp dialect:
So far this is the same as the Javascript version. The add
function has no idea that you multiplied a bunch of numbers together to create its first argument, ditto for Math/pow
.
Let’s create an add
macro instead. To keep things simple, let’s have it print its arguments and return nothing:
And here’s what you’d see if you invoked it:
The add
function is able to "see" its arguments in their entirety! Note that clojure.lang.PersistentList
is simply the type Clojure uses for its linked list data structure, so these arguments can be manipulated just like regular lists.
Crucially, you can now have the add
macro generate an implementation based on these arguments. You could run Math/abs
on every single integer that was used to construct the arguments to add
, or attempt to detect if one of the arguments has overflowed.
This was a very contrived example, but it goes to show how a programmable programming language can let you do things that are impossible in other languages.
Programmable programming languages allow you to write functions that accept code as their input, as opposed to the values that that code may evaluate to. This lets you write programs that write programs, and do many things that are impossible in other programming languages.
-
Viaweb used macros to implement RTML, a programming language end-users could use to customize their online stores. [1]
-
Clojure used macros to implement
core.async
, bringing (the equivalent of) async/await to the language with no changes to the language necessary. Imagine implementing channels in Go or async/await in Javascript with just a library!
Spreadsheets
Spreadsheets are a ubiquitous no-code tool, but they’re also legitimate programming environments. Complexity is carefully hidden or abstracted away so using spreadsheets don’t really feel like programming, making them accessible to so many more people.
Figma
Figma is a collaborative design tool that ostensibly has nothing to do with programmability. You design things in a GUI with no code in sight. However, Figma has a robust plugin system that lets you write code that has full control over the artifact being designed.
Programmability in the Modern Web
The Spectrum of Available Tools
With that introduction to the concept of programmability, let’s segment web-based tools by how programmable they are. At a high level, we think there are four buckets:
-
From-scratch tools that you use to build software & services from scratch
-
Progammable tools let you use code as the primary means to build things without the complexity of doing everything from scratch
-
Hybrid tools are fully visual but allow dropping down into code when necessary
-
No-code tools are fully visual, with no programmability
Once you start filling these buckets up[2], it quickly becomes clear that two buckets are very well-represented. If you’re using a fully visual tool with no programmability or writing something entirely from scratch, you have a ton of options to choose from!
The middle two buckets are important too, though, and they’re strangely sparse. This leads us to believe that there’s a whole segment of users – power users, developers, or simply anyone that isn’t put off by a learning curve – that today’s tools aren’t really focusing on.
No-code tools may democratize content creation on the web, but they certainly don’t democratize programming. We want more tools that nudge people towards programming, instead of leaving the true power walled off behind a GUI.
Here’s a quote from an essay that refers to this idea as end-user programming:
There is a moral imperative for end-user programming: to avoid a digital divide. A world where only a tiny elite of high priests (aka “programmers”) have control over what happens in our computing lives is concerning. This is the well from which programming literacy advocates such as Code.org draw their vigor.
More practically, the business case for end-user programming is substantial. In enterprise software, end-user programmers are sometimes referred to as “citizen developers”. Businesses of all sizes run their daily processes on tools built by non-programmer domain experts with enterprise-oriented tools like Filemaker, Microsoft Access, Force.com, and Quick Base.
If programmable tools can exist and be successful in the enterprise, why don’t we have more programmable tools that are consumer-focused? There’s this somewhat pervasive idea that consumer-grade tools have to be whittled down to the least common denominator, with the assumption that users want simplicity and minimalism above all else.
We think it’s misguided to apply this principle universally – there’s room for powerful tools that empower users by making programmability a first-class citizen.
Excel is probably the prototypical example of this: it is a reactive dataflow system, but you’ll never find any draggable widgets with little input ports and output ports you can connect.
It’s trivial to get started with Excel. It’s trivial to connect cells and get some computations going. It’s then a nice smooth transition all the way up to multidimensional analysis, and not once do you have to break out into a full programming language.
That’s what people want.
What you imagine they want is something childish and useless. To be honest, it’s simply arrogance. By assuming the user can’t code, it’s natural to assume that they think at a level of a young child and need only toys.
For fuck’s sake, Excel is 64-bit and fully parallel! Did you know that? I’ve seen 60 year old ladies from accounting produce spreadsheets that can bring a 16-core terminal server to its knees, and that spreadsheet was doing entirely legitimate, useful, productive computation. Show me the last system you built that can efficiently utilise a modern high-core CPU for a single user!
Your market is adults. They need power tools, not toys.
The word code is almost ridiculously overloaded, though. You’ve got UI code, backend code, business logic, configuration as code, and infrastructure as code. Code for persistence, routing, authentication, deployments, and monitoring.
Surely we don’t want to expose all this complexity to the end user?
Just Code
Well … no! One approach is to simply expose none of this complexity to the end-user, which is what no-code tools do.
Here’s another way to think about this. Say you write a command-line solver for Wordle and want to put it up on the internet. You’ll have to write so much code that has nothing to do with the actual solving process!
What if you could port your solver script to the internet without really worrying about anything extraneous? Well, we did, and the code is reasonably close to what we’d have written for the command-line.
Here’s a different tack: expose just some of this complexity to the end-user. The system could handle all the incidental complexity, while giving the end-user programmable control only over the bits of code that are essential to the task at hand.
There isn’t really an established name for tools that work this way. The closest thing we’ve seen is the term Just Code:
We actually need "Just Code" tooling.
The No Code movements shines a light on how bad we let things become for programmers (and aspiring ones). Building a modern full-stack website these days involves a lot of knowledge, everything from docker to react, to how to use github, npm, vscode, the command line, etc.
No Code/Low Code recognizes that most of that is "accidental complexity", and removes it. In a sense, it really lets you focus on the actual program (the "essential complexity"), in a way that "real" coding does not.
The problem with No Code is that "code" is actually important, by which I mean functions and types and values. Things that are composable and decomposable - you can look under the covers and there’s more code, or you can combine two things together to make a bigger thing. That’s as opposed to "wizards", which many (not all) No Code tools are.
We can remove the accidental complexity without removing the code though.
This resonates with us: these tools let you get rid of all the baggage and effort involved with from-scratch development except the core business logic.
No servers, deployments, backups, containers, databases, APIs, load balancing, frontends, or security: just code. We think this is a valuable approach that not many tools take, and we’re exploring the idea by building a tool that has exactly this focus. Which brings us to…
Formulate
Let’s now zoom in on a specific type of web-based tool: form-builders. These tools typically allow you to specify "logic" to make decisions based on user input. You can skip, modify, or repeat questions based on the user’s answers, perform calculations, or even perform a preprocessing step by creating derived answers.
We think this is a great fit for a programmable tool. A form-builder that allows you to specify logic using a real programming language could be powerful.
But we’re getting ahead of ourselves. What is a form?
What is a form, anyway?
Ignoring all the bells and whistles, a form is simply a tool to gather user input on the web. HTML has had a form
tag since 1995. From the original RFC:
A form is a template for a form data set and an associated method and action URI. A form data set is a sequence of name/value pair fields. The names are specified on the NAME attributes of form input elements, and the values are given initial values by various forms of markup and edited by the user. The resulting form data set is used to access an information service as a function of the action and method.
This had the monumental effect of changing HTML from a read-only medium to a read-write medium. You could now use it to accept arbitrary data from the end-user. You’d have to write some HTML code with a form
tag, set up a server of some kind to handle the form’s action
, and you were good to go!
form
tag isn’t used.
Nowadays people use forms for all sorts of things: from market research, gauging employee engagement, receiving customer feedback, and handling signups & registrations, to formal surveys, quizzes, and even to accept payments.
Here’s our take on a programmable version…
Programming Environment
A Formulate form is simply a Javascript file that exports a default function. Here’s a tiny example:
Restrictions
You can do anything you like inside this file with a few caveats:
-
You have access to Javascript-the-language, but you can’t use any Web APIs, such as the DOM or the
window
object. -
The default
Math.random()
implementation is replaced with a seeded version, so each instance of a form will see the same sequence of random values until the seed is reset, which only happens when a form is loaded for the first time, a form is completed, or when you Start Over. -
new Date()
andDate.now()
are locked in place, and return the instant at which the form was loaded. As above, this only resets when you Start Over or complete a form.
That snippet of code produces this form:
The file has access to a form
global variable that allows it to interact with the end-user. You can use it to present questions to the user, make HTTP requests, log to the browser console, or examine query parameters. It’s the only interface that the code in this file has to perform side effects of any kind.
The code executes in a sandbox inside the end-user’s browser.
The API
Here’s a quick rundown of everything you can do with the Formulate API.
Questions
This is really any form’s bread and butter. Present questions to the user and receive answers in return. The API currently allows for these six question types:
Accept a single line of text, similar to an input
field.
Accept multiple lines of text, similar to a textarea
.
User picks from options. All options are visible at once.
SelectUser picks from options. Options are hidden behind a dropdown.
RatingAccept a star rating.
PaymentAccept a payment via Stripe.
Each of these returns a Promise
, which is then fulfilled when the end-user has responded in some way.
Utilities
The API also contains some peripheral tooling to round things out; here’s a quick list:
You can use form.fetch
to make HTTP requests at runtime, and have your form dynamically alter itself in response.
Use form.log
or form.error
to log data to the browser console.
Use form.query
to retrieve a single value from the URL query string.
The Lodash utility library is always available for cases where the JS standard library just isn’t cutting it.
What can I build with this thing?
Those are the basic building blocks available so far. We haven’t explored the space of things that they enable you to build too much yet, but at least theoretically, the sky’s the limit!
Here’s a small list of ideas we’ve had so far that focus on use cases that programmability makes simpler (or even possible at all):
-
Each respondent sees a percentage of a large pool of questions
-
Each question is a random choice between multiple variants
-
Only 50% of respondents see a given question
-
Some form of feature flagging, where you instantly enable or disable certain questions
form.fetch
-
Load the entire set of questions dynamically from a remote file or database
-
Build a quiz about that day’s news (or Rick & Morty)
-
Personalize a form based on weather data in the respondent’s location
-
Create a poll that uses a list of restaurants nearby that are currently open
-
Build charts based on ongoing responses
-
Show different questions to premium users, regular users, and non-users
-
Generate coupon codes on the fly
-
Respondents pick their favorite books, music, podcasts, or even tweets
-
Custom validation and data cleaning: ask the respondent to redo a question until it matches your schema/format exactly
-
Maintain a user-visible running total (or score)
-
Respondents verify their email address with an OTP
Next Steps
And that’s everything we’ve got so far! As with any programmable tool, we’re sure Formulate can be used for a ton of things we aren’t even thinking about yet.
Here’s what we are thinking about, at least for the near future:
We’ve only spent roughly a month’s worth of evenings and weekends working on Formulate so far, and feature-wise there are a ton of gaps. Customization options are limited, response data is only available as a CSV download, there are no real user accounts, the UI can be inconsistent (especially on small screens), and you can’t use it from the command-line.
Here’s a quick example: an integration with Google Sheets could be useful in and of itself, but we want to allow controlling this behavior via the form
interface.
This should let you do things like picking a target sheet/cell or applying custom formatting based on a user’s responses.
We want to fill in these gaps one by one, while also being thoughtful about making things programmable where possible.
Formulate feels a lot more ergonomic to us than other form-builders we’ve used in the past, but it’s still very much in its infancy. We don’t know if anyone is going to be interested in a tool like this, all we really know is that we really like using it so far.
We want to have conversations with anyone who has opinions on this sort of thing to refine Formulate’s core premise until it’s perfect. If you’d like to help shape the direction this takes, please get in touch!
Formulate’s only interface is code. This is a good starting point, but we’d really like to evolve towards something more like Excel, where you have a simpler interface presented as the default, with a fully programmable layer underneath.
We don’t have this simpler interface yet, and we aren’t quite sure what it should look like. We’re confident we don’t want a traditional drag-n-drop GUI – there are plenty of those already – but we haven’t gone much further here yet.
One initial idea is to use a plaintext interface, similar to Inform 7 or writing slide decks in Markdown. More ideas are welcome!
Conclusion
Nothing! Formulate is a side project that we have limited time to work on, so we’re focused on the core feature set, not pricing.
With that said, the infrastructure behind this thing is not free, so we’re going to want to implement some sort of pricing model in the future to cover costs.
Formulate started as an "is this even possible to build" hobby project, but we’ve since become convinced that that current state of web-based tooling leaves a lot of gaps for developers and power users. We hope you’ll give Formulate a shot and tell us what you think.
Squinting ever-so-slightly, you could even use Formulate for things that are decidedly not forms. Treating it more generally as an abstraction for user input in a browser, you could use it to build games, tiny web apps, or maybe even teach people to code!
We don’t have all the answers yet, but we like the shape this has taken so far, and we’re excited to keep going!
Finally, a couple of resources: we’re on Twitter, there’s a discussion forum if you’re stuck or have questions, you can find API documentation here, and here’s a tiny gallery of example forms to get you started.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK