The magical world of Particles with React Three Fiber and Shaders
source link: https://blog.maximeheckel.com/posts/the-magical-world-of-particles-with-react-three-fiber-and-shaders/?ref=sidebar
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
The magical world of Particles with React Three Fiber and Shaders
November 8, 2022 / 26 min read /
232 Likes •
17 Replies •
36 Mentions
Since writing The Study of Shaders with React Three Fiber, I've continued building new scenes to perfect my shader skills and learn new techniques to achieve even more ambitious creations. While shaders on their own unlocked a new realm of what's possible to do on the web, there's one type of 3D object that I've overlooked until recently: particles!
Whether it's to create galaxies, stars, smoke, fire, or even some other abstract effects, particles are the best tool to help you create scenes that can feel truly magical 🪄.
However, particles can also feel quite intimidating at first. It takes a lot of practice to get familiar with the core concepts of particle-based scenes such as attributes or buffer geometries and advanced ones like combining them with custom shaders or using Frame Buffer Objects to push those scenes even further.
In this article, you will find all the tips and techniques I learned regarding particles, from creating simple particle systems with standard and buffer geometries to customizing how they look, controlling their movement with shaders, and techniques to scale the number of particles even further. You'll also get a deeper understanding of attributes, a key shader concept I overlooked in my previous blog post that is essential for these use cases.
👉 This article assumes you have basic knowledge about shaders and GLSL, or read The Study of Shaders with React Three Fiber.
The GLSL code in the demos will be displayed as strings as it was easier to make that work with React Three Fiber on Sandpack.
To learn more on how to import .glsl
files in your React project, check out glslify-loader.
An introduction to attributes
Before we can jump into creating gorgeous particle-based scenes with React Three Fiber, we have to talk about attributes.
What are attributes?
Attributes are pieces of data associated with each vertex of a mesh. If you've been playing with React Three Fiber and created some meshes, you've already used attributes without knowing! Each geometry associated with a mesh has a set of pre-defined attributes such as:
- ArrowAn icon representing an arrowThe position attribute: an array of data representing all the positions of each vertex of a given geometry.
- ArrowAn icon representing an arrowThe uv attribute: an array of data representing the UV coordinates of a given geometry.
These are just two examples among many possibilities, but you'll find these in pretty much any geometry you'll use. You can easily take a peek at them to see what kind of data it contains:
Logging the attributes of a geometry
You should see something like this:
Screenshot showcasing the output printed when logging the attributes of a geometryIf you're feeling confused right now, do not worry 😄. I was too! Seeing data like this can feel intimidating at first, but we'll make sense of all this just below.
Playing with attributes
This long array with lots of numbers represents the value of the x, y, and z coordinates for each vertex of our geometry. It's one-dimensional (no nested data), where each value x, y, and z of a given vertex is right next to the ones from the other vertex. I built the little widget below to illustrate in a more approachable way how the values of that position array translate to points in space:
As you can see, to read this array, we need to read values 3 by 3 simply because our vertices have three values. To read the UV attribute array, however, we need to read the values 2 by 2, as UV coordinates only have two values, x and y.
We will see later in this article how to define attributes and how we can tell our renderer how to "read" the data to build custom geometries.
Now that we know how to interpret that data, we can start having some fun with it. You can easily manipulate and modify attributes and create some nice effects without the need to touch shader code.
Below is an example where we use attributes to twist a boxGeometry
along its y-axis.
We do this effect by:
- ArrowAn icon representing an arrowCopying the original
position
attribute of the geometry.
- ArrowAn icon representing an arrowLooping through each value of the array and applying a rotation.
- ArrowAn icon representing an arrowPass the newly generated data to the geometry to replace the original
position
attribute array.
Attributes with Shaders
I briefly touched upon this subject when I introduced the notion of uniforms in The Study of Shaders with React Three Fiber but could not find a meaningful way to tackle it without making an already long article even longer.
We saw that we use uniforms to pass data from our Javascript code to a shader. Attributes are pretty similar in that regard as well, but there is one key difference:
- ArrowAn icon representing an arrowData passed to a shader via a uniform remains constant between each vertex of a mesh (and pixels as well)
- ArrowAn icon representing an arrowData passed via an attribute can be different for each vertex, allowing us to more fine-tuned controls of our vertex shader.
You can only pass attributes to the vertex shader! If you want to use them in a fragment shader, you will need to pass the data using a varying.
Diagram illustrating how to pass the attributes from a geometry from the vertex shader to the fragment shader using varyings.You can see that attributes allow us to control each vertex of a mesh, but not only! For particle-based scenes, we will heavily rely on them to:
- ArrowAn icon representing an arrowposition our particles in space
- ArrowAn icon representing an arrowmove, scale, or animate our particles through time
- ArrowAn icon representing an arrowcustomize each particle in a unique way
That is why it's necessary to have a somewhat clear understanding of attributes before getting started with particles.
Particles in React Three Fiber
Now that we know more about attributes, we can finally bring our focus to the core of this article: particles.
Our first scene with Particles
Remember how we can define a mesh as follows: mesh = geometry + material? Well, that definition also applies to points, the construct we use to create particles:
points = geometry + material
The only difference at this stage is that our points will use a specific type of material, the pointsMaterial.
You can read more about those constructs by heading to the corresponding section in the Three.js documentation:
- ArrowAn icon representing an arrow
- ArrowAn icon representing an arrow
That is also where you'll find all the options documented, as I may skip detailing some of those in this article.
Below you'll find an example of a particle system in React Three Fiber. As you can see, we're creating a system in the shape of a sphere by using
- ArrowAn icon representing an arrow
points
- ArrowAn icon representing an arrow
sphereGeometry
for our geometry - ArrowAn icon representing an arrow
pointsMaterial
for our material
With pointsMaterial
you can:
- ArrowAn icon representing an arrowmake your particles bigger or smaller using the
size
prop. - ArrowAn icon representing an arrowmake distant particles look smaller than closer particles using the
sizeAttenuation
prop.
Now you may ask me: this is great, but what if I want to position my particles more organically? What about creating a randomized cloud of particles? Well, this is where the notion of attributes comes into play!
Using BufferGeometry and attributes to create custom geometries
In Three.js and React Three Fiber, we can create custom geometries thanks to the use of:
- ArrowAn icon representing an arrow
bufferGeometry
- ArrowAn icon representing an arrow
bufferAttribute
- ArrowAn icon representing an arrowour newly acquired knowledge of attributes 🎉
When working with Particles, using a bufferGeometry
can be really powerful: it gives us full-control over the placement of each particle, and later we'll also see how this lets us animate them.
Let's take a look at how we can define a custom geometry in React Three Fiber with the following code example:
Custom geometry with bufferGeometry and bufferAttribute
In the code snippet above, we can see that:
- ArrowAn icon representing an arrowWe are rendering a
bufferGeometry
as the geometry of our points. - ArrowAn icon representing an arrowIn this
bufferGeometry
, we're using thebufferAttribute
element that lets us set the position attribute of our geometry.
Now let's take a look at the props
that we're passing to the bufferAttribute
element:
- ArrowAn icon representing an arrow
count
is the total number of vertex our geometry will have. In our case, it is the number of particles we will end up rendering. - ArrowAn icon representing an arrow
attach
is how we specify the name of our attribute. In this case, we set it asattributes-position
so the data we're feeding to thebufferAttribute
is available under theposition
attribute. - ArrowAn icon representing an arrow
itemSize
represents the number of values from our attributes array associated with one item/vertex. In this case, it's set to3
as we're dealing with theposition
attribute that has three components x, y, and z.
I'd recommend reading the documentation on the attribute notation.
I did not do it and lost a couple hours due to a silly mistake the first time I tried custom geometries 🤦♂️
well... that's 2 hours of my life I won't get back https://t.co/go8qCC5cVG
3:14 PM - Aug 30, 2022Now when it comes to creating the attributes array itself, let's look at the particlePositions
array located in our particle scene code.
Generating a position attribute array
- ArrowAn icon representing an arrowFirst, we specify a
Float32Array
with a length ofcount * 3
. We're going to rendercount
particles, e.g. 2000, and each particle has three values (x, y, and z) associated with its position, i.e. *6000 values in total. - ArrowAn icon representing an arrowThen, we create a loop, and for each particle, we set all the values for x, y, and z. In this case, we're using some level of randomness to position our particles randomly.
- ArrowAn icon representing an arrowFinally, we're adding all three values to the array at the position
i * 3
withpositions.set([x,y,z], i*3)
.
The code sandbox below showcases what we can render with this technique of using custom geometries. In this example, I created two different position attribute arrays that place particles randomly:
- ArrowAn icon representing an arrowat the surface of a sphere
- ArrowAn icon representing an arrowin a box, which you can render by changing the
shape
prop tobox
and hitting reload.
We can see that using custom geometries lets us get a more organic render for our particle system, which looks prettier and opens up way more possibilities than standard geometries ✨.
Customizing and animating Particles with Shaders
Now that we know how to create a particle system based on custom geometries, we can start focusing on the fun part: animating particles! 🎉
There are two ways to approach animating particles:
- ArrowAn icon representing an arrowUsing attributes (easier)
- ArrowAn icon representing an arrowUsing shaders (a bit harder)
We'll look at both ways, although, as you may expect, if you know me a little bit through the work I share on Twitter, we're going to focus a lot on the second one. A little bit of challenge never hurts!
Animating Particles with attributes
For this part, we will see how to animate our particles by updating our position attribute array on every frame using the useFrame
hook. If you've animated meshes with React Three Fiber before, this method should be straightforward!
We just saw how to create an attributes array; updating it is pretty much the same process:
- ArrowAn icon representing an arrowWe loop through the current values of the attributes array. It can be all the values or just some of them.
- ArrowAn icon representing an arrowUpdate them.
- ArrowAn icon representing an arrowAnd finally, the most important: set the
needsUpdate
field of our position attribute totrue
.
If you forget the last step, your scene will remain static!
Animate particles via attributes in React Three Fiber
The scene rendered below uses this technique to move the particles around their initial position, making the particle system feel a bit more alive ✨
Despite being the easiest, this method is also pretty expensive: on every frame, we have to loop through very long attribute arrays and update them. Over and over. As you might expect, this becomes a real problem as the number of particles grows. Thus it's preferable to delegate that part to the GPU with a sweet shader, which also has the added benefit to be more elegant. (a totally non-biased opinion from someone who dedicated weeks of their life working with shaders 😄).
How to animate our particles with a vertex shader
First and foremost, it's time to say goodbye to our pointsMaterial
👋, and replace it with a shaderMaterial
as follows:
How to use a custom shaderMaterial with particles and a custom buffer geometry
As we learned in The Study of Shaders with React Three Fiber, we need to specify two functions for our shaderMaterial
:
- ArrowAn icon representing an arrowthe fragment shader: this is where we'll focus on the next part to customize our particles
- ArrowAn icon representing an arrowthe vertex shader: this is where we'll animate our particles
For this example, we're going to make our particles rotate. Some good folks worked on some GLSL packages to abstract these functions away for us like dmnsgn/glsl-rotate.
You can load these functions on your projects using glsify-loader. The code sandbox below will have the code of this project copied over for simplicity.
Vertex shader code that applies a rotation along the y-axis
As you can see in the snippet above, when it comes to the code, animating particles using a shader is very similar to animating a mesh. With the vertex shader, you get to interact with the vertices of a geometry, which are the particles themselves in this use case.
Since we're there, let's iterate on that shader code to make the resulting scene even better: make the particles close to the center of the sphere move faster than the ones on the outskirts.
Enhanced version of the previous vertex shader
Which renders as the following once we wire this shader to our React Three Fiber code with a uTime
and uRadius
uniform:
How to change the size and appearance of our particles with shaders
This entire time, our particles were simple tiny squares, which is a bit boring. In this part, we'll look at how to fix this with some well-thought-out shader code.
First, let's look at the size. All our particles are the same size right now which does not really give off an organic vibe to this scene. To address that, we can tweak the gl_PointSize
property in our vertex shader code.
We can do multiple things with the point size:
- ArrowAn icon representing an arrowMaking it a function of the position with some Perlin noise
- ArrowAn icon representing an arrowMaking it a function of the distance from the center of your geometry
- ArrowAn icon representing an arrowSimply making it random
Anything is possible! For this example, we'll pick the second one:
When we replaced our pointsMaterial
with a shaderMaterial
we lost the `sizeAttenuation* prop.
If you want to easily reproduce it in a vertex shader, the code to add is:
Now, when it comes to the particle pattern itself, we can modify it in the fragment shader. I like to make my particles look like tiny points of light that we can luckily achieve with a few lines of code.
Fragment shader that changes the appearance of our particles
We can now make the colors of the particles a parameter of the material through a uniform and also make it a function of the distance to the center, for example:
Enhanced version of the previous fragment shader
In the end, we get a beautiful set of custom particles with just a few lines of GLSL sprinkled on top of our particle system 🪄
For a better effect, set the blending
prop of the shaderMaterial
to THREE.AdditiveBlending
. This allows particles that are superposed to add their color to one another and create a beautiful staturated look.
Going beyond with Frame Buffer Objects
What if we wanted to render a lot more particles onto our scene? What about 100's of thousands? That would be pretty cool, right? With this advanced technique I'm about to show you, it is possible! And on top of that, with little to no frame drop 🔥!
This technique is named Frame Buffer Object (FBO). I stumbled upon it when I wanted to reproduce one of @winkerVSbecks attractor scenes from his blog post Three ways to create 3D particle effects.
Long story short, I wanted to build the same attractor effect but with shaders. The problem was that in an attractor, the position of a particle is dictated by its previous one, which doesn't work by just relying on the position attributes and a vertex shader: there's no way to get the updated position back to our Javascript code after it's been updated in our vertex shader and feed it back to the shader to calculate the next one! Thankfully, thanks to using an FBO, I figured out a way to render this scene.
It was mainly thanks to this Stackoverflow answer that I figured out the solution and learned the existence of FBO.
How does a Frame Buffer Object work with particles?
I've seen many people using this technique in Three.js codebases. Here is how it goes: instead of initiating our particles positions array and passing it as an attribute and then render them, we are going to have 3 phases with two render passes.
- ArrowAn icon representing an arrowThe simulation pass. We set the positions of the particles as a Data Texture to a shader material. They are then read, returned, and sometimes modified in the material's fragment shader (you heard me right!).
- ArrowAn icon representing an arrowCreate a
WebGLRenderTarget
, a "texture" we can render to off-screen where we will add a small scene containing our material from the simulation pass and a small plane. We then set it as the current render target, thus rendering our simulation material with its Data Texture that is filled with position data. - ArrowAn icon representing an arrowThe render pass. We can now read the texture rendered in the render target. The texture data is the positions array of our particles, which we can now pass as a
uniform
to our particles'shaderMaterial
.
It may sound counter-intuitive for a fragment shader to store/return position data when, so far, we mainly used it for colors. If you think about it, the meaning of the data doesn't really matter, it's more about its shape:
- ArrowAn icon representing an arrowColors in a fragment shader are a
vec4
for the R, G, B, and A color components. - ArrowAn icon representing an arrowIn this case, we're also passing a
vec4
for x, y, and z and a constant value1.0
for the last component that we do not need.
In the end, we're using the simulation pass as a buffer to store data and do a lot of heavy calculations on the GPU by processing our positions in a fragment shader, and we do that on every single frame. Hence the name Frame Buffer Object. I hope I did not lose you there 😅. Maybe the diagram below, as well as the following code snippet will help 👇:
Diagram illustrating how the Frabe Buffer Objects allows to store and update particles position data in a vertex shader and then be read as a texture.Setting up a simulation material
Setting up an FBO with a simulation material in React Three Fiber
Here I'm relying on two tools provided by the pmndrs team:
- ArrowAn icon representing an arrowthe
useFBO
hook from @react-three/drei to set up my render target. - ArrowAn icon representing an arrowthe
createPortal
from @react-three/fiber to render the necessary objects used by my render target off-screen
If you want to learn more about Three.js render targets, you should check out this introduction article
Creating magical scenes with FBO
To demonstrate the power of FBO, let's look at two scenes I built with this technique 👀.
The first one renders a particle system in the shape of a sphere with randomly positioned points. In the simulationMaterial
, I applied a curl-noise to the position data of the particles, which yields the gorgeous effect you can see below ✨!
In this scene, we:
- ArrowAn icon representing an arrowrender
128 x 128
(the resolution of our render target) particles. - ArrowAn icon representing an arrowapply a curl noise to each of our particles in our simulation pass.
- ArrowAn icon representing an arrowpass all that data along to the
renderMaterial
that takes care to render each vertex with that position data and also the particle size using thegl_pointSize
property.
I kept the number of particles "low" on purpose, so I could be sure it performs well on most computers, but I'd invite you to fork this scene and increase the resolution of the render target for an even more impressive effect!
On my laptop, a 2020 M1 Macbook Pro, I can push this demo way over 1 million particles 🤯.
Finally, one last scene, just for fun! I ported to React Three Fiber a Three.js demo from an article written by @nicoptere that does a pretty good job at deep diving into the FBO technique.
In it, I pass not only one but two Data Textures:
- ArrowAn icon representing an arrowthe first one contains the data to position the particles as a box
- ArrowAn icon representing an arrowthe second one as a sphere
Then in the fragment shader of the simulationMaterial
, we use GLSL's mix
function to alternate over time between the two "textures" which results in this scene where the particles morph from one shape to another.
Conclusion
From zero to FBO, you now know pretty much everything I know about particles as of writing these words 🎉! There's, of course, still a lot more to explore, but I hope this blog post was a good introduction to the basics and more advanced techniques and that it can serve as a guide to get back to during your own journey with Particles and React Three Fiber.
Techniques like FBO enable almost limitless possibilities for particle-based scenes, and I can't wait to see what you'll get to create with it ✨. I couldn't resist sharing this with you in this write-up 🪄. Frame Buffer Objects have a various set of use cases, not just limited to particles that I haven't explored deeply enough yet. That will probably be a topic for a future blog post, who knows?
As a productive next step to push your particle skills even further, I can only recommend to hack on your own. You now have all the tools to get started 😄.
Already 248 awesome people liked, shared or talked about this article:
Tweet about this post and it will show up here! Or, click here to leave a comment and discuss about it on Twitter.
Do you have any questions, comments or simply wish to contact me privately? Don’t hesitate to shoot me a DM on Twitter.
Have a wonderful day.
Maxime
Subscribe to my newsletter
Get email from me about my ideas, frontend development resources and tips as well as exclusive previews of upcoming articles.
2022-11-08T08:00:00.000+01:00
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK