Fundamentals all the way down
source link: https://changelog.com/jsparty/283
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Transcript
Changelog
Click here to listen along while you enjoy the transcript. š§
Hoy-hoy. Welcome to another exciting JS Party. Iām your host this week, Nick Nisi, and I am joined by two wonderful regulars, and that is Kball - Kball, howās it going?
Going alright, coming in hot.
[laughs] Yes, as always. And also Jerodās here. Jerod, howās it going?
Itās going great, because Iām here with you, Iām here with Kball, and my ears have this ringing, and Iām not sure why.
Thatās just my voiceā¦
I think it might have been your fault.
Itās the sound of angels.
And that angelic voice that youāve heard is Austin Gil joining us again. Austin, howās it going?
Hey. Iām doing well, living the good summer life here in Portland, taking a break from camping, and river floats, and bike rides to come hang out with you all. Itās such a pleasure.
Good to have you.
River float sounds goodā¦
Come by sometime. I will take you. I will make sure anyone on the show, anyone watching, anyone listening - come visit, I will show you a good time in Portland.
No promises there, but I can confirm that I will see you in person in Lincoln, Nebraska next month. So thatās pretty exciting.
Thatās right. Iām very excited.
Do you do river floats there?
Noā¦ Well, I think there is a riverā¦
We have rivers. Come on, Nick, donāt sell us short.
We do have ā do we have a river in Lincoln?
Oh, Lincoln. No. Nothing good ever came out of Lincoln. Just kiddingā¦ Sort of.
[laughs] But very exciting. Weāre gonna meet up in person for the first time, and - I just got my ticket yesterday.
Where are you guys gonna meet up?
Lincoln, it sounds likeā¦
Yeah, everybodyās going to Lincoln.
But why? But why? Is there a reason?
I donāt know, Nebraska, but Iām guessing thatās like five people, or something. Itās easy to find.
Well, yeah. Iāll be there for Nebraska Codes. It should be a fun conference. Iāll be speaking there, so if anyone wants to come find me - yeah, letās be friends.
Now, Austin, we had you on - letās seeā¦ It was quite a while ago; I think it was episode 243, and we were talking about HTML access without JavaScript from your camera. And that was a fascinating article. So we were kind of talking about what youāve been up to lately, and it sounds like a little more digging into the file system APIā¦ Why donāt you catch us up on what youāve been doing?
Yeah, so I donāt know why I get into these deep-dives into fundamentals ā well, we use fundamentals as a loaded termā¦ But yeah, into like fundamentals of whatās available in the programming languages, and frameworks, and tools, and platforms that I useā¦ So I did a deep-dive on forms, and then that led me into accessing files with just HTML, or accessing camera with just HTMLā¦ And then recently, I have been on this sort of quest from God on trying to promote people to understand fundamentals and use fundamentals, as opposed to leaning too heavily on libraries, and frameworks, and tools like that.
And I was on a conversation recently with JavaScript Jabber guys, and we talked about just the fundamentals, and rehashed this familiar conversation of āShould you focus on fundamentals, or should you focus on frameworks and libraries and tools?ā and it became a very nebulous conversationā¦ So Iād like to keep promoting peopleās understanding of fundamentals, without getting too into like heady/space/galaxy brain stuffā¦ You know, maybe get into some more practical information today.
Where does the Galaxy brain land with regards to fundamentals?
Well, I just think ā I guess nebulous is the best term. Itās a loaded term that doesnāt have a whole lot of clear definition of what youāre talking about when you mean the fundamentals. I think the best that Iāve come to is saying that the fundamentals are sort of the lower-level knowledge that you need in order to explain the topic that youāre discussing about. So when weāre talking about fundamentals, I think a good framework to understand it is to say take a tool like Next.js - Next.js has a whole bunch of things that it makes available just within its sort of ecosystem, and building a Next.js application requires understanding those fundamental principles of that tool. But Next.js is built on top of React, and there are boundaries between what Next offers and what React offers. And React is like the lower-level primitive of that metaframework. And then React is built on top of JavaScript, so thereās fundamental knowledge that you need to understand the separation of concerns between JavaScript and Reactā¦ And then when youāre with JavaScript, then you have to understand the platform, andā¦
Itās turtles all the way down. We could take this down and say, āOkay, now we need to understand interpreters, and interpreted language, and that means we need to understand machine code, and compilersā, and like all these different layers.
But that is what Iām saying. But you have to find where you live in the layers, right? Like, where are you right now, and the work that youāre doing - you need to understand the fundamentals of that. You donāt need to go all the way down to ones and zeros, to understand everything. But say, āWhat is the code that Iām writing doing, and what of that code lives within the current stack, or the current layer of the stack, and what belongs to the next layer down?ā
And I think at some point you do have to get down to like lower-level things, like HTTP, the protocols, and the platforms. And then at that point ā
I mean, some of this comes to how leaky are your abstractions.
Exactly, yes. Yes.
[06:01] The web platform and JavaScript provide pretty solid abstractions. Itās pretty rare that those leak and you have to go lower than that if youāre writing for the web. Next, or Nuxt, has some leaky spots.
Yeah, yeah. And thatās kind of what Iām promoting, is mostly the curiosity to take a moment and check your knowledge of understanding where those abstractions are, and who is responsible for those. And it just helps you write better software.
Please allow me to make this more abstract by introducing a metaphor, because I feel like this would be a great episode if we never let Austin get to his actual fundamental topic, and we just keep goingā¦
[laughs] This is exactly how the last conversation wentā¦ Itās great.
So as a sports ball coach - you know, we coach basketball. And when you get the kids in the room, and you give them a ball, and you say āOkay, itās time for fundamentals.ā And Kball, we teach them dribbling, and passing, and shooting, and defense. Thatās basically the fundamentals. Weāre not gonna teach them gravity, or how to tie their shoes, or stuff like this, which actually are lower-level than the fundamentals, but theyāre not the fundamentals of basketball.
Now, having said that, what they like to do though is not come in and learn how to dribble; what they like to do is come to half court and just chuck the ball at the rim.
Scrimmage!
Yeah, see if they can get a half court shot to go in. Thatās what they want to do. And so thereās your tie back to this desire to go straight to the end game, which is what we do a lot of times in web development. Itās like āWell, Iām trying to get things done, and so Iām gonna grab Next.js, and Iām gonna maybe skip the dribbling and the passing and the shooting fundamentals. I can backfill those later.ā Austin is here to teach us how to dribble.
I think one of the key questions there is how you make dribbling fun, right? Some people - Iām gonna run with your metaphor, except for me itās coaching soccer. Iāve been coaching soccer, rather than basketball.
Okay, fair enough.
But you still have dribbling. We have that in common.
So there are kids that they can map that āIf I get better at this, Iām going to be better at that.ā I think the same is true for adults. There are adults who can say, āOkay, if I focus on this thing, thatās going to eventually help me with that, so itās worth doing that.ā And there are kids where if you have them out there doing drills, they check out. Theyāre not getting anything from that fundamentals work, because theyāre like āI donāt see ā whatās the point? This is boring. Iām gonna go chase butterflies.ā But if you get them in something that feels realā¦ You know, a little scrimmage, maybe you narrow it down to like a 2-1-2, so they get lots of touches on the ball, they get lots of opportunity to start to move their body in these patternsā¦ And you can give them little nudges here and there and say, āOkay, there you want to kick it a little bit more like this. Or that was a good opportunity to do a little bit more dribblingā, but itās within a framework that looks a lot more like the real thing.
So Iām going to pull this back then to Austin - where is your 2-1-2? Whereās your 2-1-2 when it comes to web fundamentals?
Wow. Tie that back, Austin.
Uhhhā¦ [laughter] I donāt know if I can answer that. Iām just gonna deflect that and say that ā I think that you touched on a really good point, that I wanted to bring back, which is being able to see your developmental path. And if you want to get better at software engineering, then I think understanding the fundamentals helps with that, because while youāre writing code, if you can periodically check in with yourself and say, āHow much of this process or this functionality that Iām building do I understand what a tool or library or framework is doing?ā, to be able to explain and say, āOkay, itās definitely good that Iām using this tool, because if I didnāt, Iād know what it would take to rebuild the same functionality, and thereās too much work. Like, that abstraction is worth keeping.ā And otherwise, you can say āThat abstraction may not be worth keepingā, because that can paint you into a corner. That can be limiting to working with the API that they provide, and not be able to take advantage of certain features that are at that lower level, that could be introducing more blow into client-side tooling, because youāre including a library that you really only need a small subset of the fundamentals, of what the platform can provide, that this library builds on top of, and introduces more work to download; libraries can also introduce more maintenance headache, can introduce more security vulnerabilities in addition to that bloat, can introduce that story where you come back to a project after six months and the upgrade story is a nightmare, because Node version compatibility is broken, or things like thatā¦ Itās just itās kind of a nightmare.
[10:43] It doesnāt have to be that wayā¦
It doesnāt have to be that wayā¦
But it is that way.
But it kind of is that way. Yeah. Like, itās been that way for my career.
[laughs] Letās be real.
Maybe it doesnāt in the future, butā¦ I donāt know, thereās actually ā well, thatās a story for another time. But I think one of the most valuable skills that is underrated in terms of using tooling, or opting into using tooling, is as soon as you start relying on frameworks to do the work for you, you give up transferable knowledge. And when you work at a lower level - this is a JavaScript podcast. So when I have ā
Ostensibly.
Yeah. [laughs] When I have code thatās like - letās say a random string generator to generate IDs for me; if Iām using that from a package that I rely on npm, or a tool thatās available in like Solid.js, all of a sudden I have a dependency on that, and I canāt take that code. Itās not portable to take to other projects that are built on top of different tooling, and itās not knowledge that I can even transfer myself. So I like to have a snippet library in my GitHub thatās like all of these little low-level things that I donāt need a library for. I can copy off my GitHub, paste it into a different project, it works the same, as long as Iām in a JavaScript environment. Thereās a lot of value there for me.
Iām going to challenge your premise a little bit on this. This is just pushing a little bit here, which is - letās take the example of component-driven development. Component-driven development - the industry has pretty much normalized on that is the most productive way to do UI development. You break things down into components, you subdivide them in that way, and you do that. And there are a slew of different frameworks that allow you to do this in different ways, but the sort of raw platform in JavaScript does not have a good solution for this. Web Components are *beep*. Theyāre terrible. Theyāre not a valuable way to do this, because one of the key productivity boosts of this component-driven development is thinking about your UI in a declarative manner. And Web Components, for reasons that I do not understand, chose to maintain an imperative programming model.
So where Iām going to this is - if you invest time and energy in learning how to do component-driven development using one of these declarative frameworks, whether it is React, whether it is Vue, whether it is Angular, whether it is Lit, whether it is one of these other things, abstractions on top of the platform, that understanding, that model - while the framework details will vary, and that piece is not as portable, the mental model for how you do that development and do that breakdown is, and it is something that is going to be more valuable to you in moving from project to project, than if you had tried to do everything from scratch without that mental model.
Yeah, totally agree. And thatās the flip side of the coin, that I think understanding fundamentals informs your decision when you need to reach for a higher-level abstraction. Because if you say that everything should be done with the fundamentals, I think thatās going too far into the other direction. And then you say, āOkay, do everything just with vanilla JavaScript.ā And then you say, āOkay, well, Iām going to use whatās available on the platform. Maybe itās Web Components, or whatever, but they donāt provide good reactivity models.ā So then Iām going to build some sort of reactivity system on top of proxies, or something along those lines, maybe reinvent signals, because thatās pretty cool, and thatās the new hotnessā¦ And before you know it, we have reinvented one of these frameworks that already exist, but youāve created a crappier version.
[14:24] So that is what you want to avoid, but again, I think the knowledge of understanding what it is that these higher-level abstractions are doing at a lower level, and understanding what theyāre doing can inform your decision to say, āWhat does my project need?ā and does that need surpass a barrier that says āLet me reach for an abstraction, let me not reinvent the wheelā?
I think it could be argued, Kball, just to continue down this line of conversation, that at this point, a component ā itās like weāve identified a new fundamental of web development with components. Maybe itās higher level than we previously were, but yet itās so fundamental to the way that we do things now that you are actually learning fundamentals, even though youāre learning them through this framework proxy.
It depends on the framework, right? Because if itās React, then yeah, youāre doing a lot of fundamental JavaScript maps, for exampleā¦ But if itās something that has a less fundamental templating library, like Angular - but I know Angular has changed a lot, so I donāt know if I can say that, butā¦ And how is ngf tags going to help meā¦?
Yeah, well, and thatās ā that is actually kind of an interestingā¦ Thereās a layering of knowledge here, which is thereās syntax, and tactical details of the APIs that youāre using in the frameworks, like how those are working, and then thereās an architectural or mental model level. So if youāre using Angular, or React, or Vue, or Svelte, or Qwik, or we can go on and on and onā¦ Like, this has become the norm, and all of these different things are doing it. They have a set of things that are detailed and specific to their framework, but there is also this architectural fundamental, as you say, Jerod, that is the same throughout all of them.
Yeah. Thereās your transferable knowledge right there.
Thereās your transferable knowledge. Exactly. And I would actually argue that those fundamentals are more important than the details of the APIs youāre using, whether youāre using a bare JavaScript API for file upload, versus youāre using something else for upload. So the medium that youāre using to learn it in is less important than that higher-level concept. And another place this shows up is programming languages. So for most software development jobs, you donāt need to have prior experience in the specific programming language that is there. You can pick that up.
Sure you do. Itās right there on the job description.
I mean, yeah, there are people who do that. But my premise would be, for example ā I mean, there are differences among different classes of programming languages, but if I have a job that isā¦ Our applicationās written in Python, say, and youāve done Node, or youāve done Ruby, I think you can pick up Python without much difficulty, because the fundamental understanding of how does a dynamic scripting language, that has object-oriented pieces and functional pieces work, is something that you can get in both of those other environments. And that fundamental knowledge is transferable, and you will learn syntax. And there are - and this comes to kind of your point, I think, Austin - some places where there are real differences between these frameworks, and at some level of complexity or at some level ofā¦ There are some projects where that nuanced difference does matter, and understanding which parts are fundamental and which parts are specific will be important to you at that pointā¦ But I would argue that thatās a pretty small minority of projects.
[18:10] So what youāre saying ultimately is āIt depends.ā
It depends. And this of course, was ā [laughter] So before we started recording, Kball didnāt want to rehash the fundamentals versus framework thing, and then I wanted to avoid the whole nebulous topic too much aboutā¦ We were in galaxy brain territory of just āNone of this makes sense.ā
So weāre doing both.
Yes, weāre doing both. Hopefully, all three.
Weāre just trolling you, Austin. This is just a long roll. Itās all this is. I mean, what does this have to do with uploading files, guys?
Exactly. So I do want to finish with thisā¦ The reason I say that fundamentals are a loaded term is because a lot of people conflate it with the underlying platform, or like vanilla JavaScript, or like the DOM, or whatever. And there is a distinction there. You donāt have to understand every single available method on inputs to know how to use them, right? You donāt need to understand that HTML can access the camera. Probably not. Rarely ever going to need that. But again, I think itās just a matter of understanding the lower-level concepts, and stopping and just ensuring that youāre taking moments in your day to stop and check your knowledge, and check your understanding of what it is that youāre doing.
So Kball, to your point about components, and that being the important thing to learn - I think at a lower level you can ā back to the full stack of knowledge thing, you might start at Next, and then move down to React, and component architecture, like Nick was saying, is a fundamental piece of knowledge within or below React. If you take that, you can swap out React, and you can take that fundamental knowledge to Vue, to Solid, to Qwik. And there are going to be some differences, and understanding those differences is the implementation detail of the next level up.
Not to dive into that too much, but then those differences also can shape how you think about those individual components.
For example, if youāre trying to avoid massive rewrites within a virtual DOM environment, you might break up into smaller components. Whereas maybe another framework better manages that, and doesnāt require you to do that, so you donāt have to break it up nearly as far.
Yeah. Itās interesting, because Iāve shifted to working with Solid a lot lately, and the way it renders things is totally different than React.
And that is fundamental knowledge you need for that tooling, because then you canāt build good applications without understanding what itās doing below, under the surface.
So it is fundamentals all the way down.
It is, it is, ultimately.
So letās bring this back to maybe a more practical, tangible topicā¦
No, no! Quick, someone interject something. Basketball, yes. [laughter]
Well, I am kind of interestedā¦ So you have dug into file uploads, and so I think that is a useful thing. Can we contrast doing bare file uploads using platform technology only, versus using a library that provides some abstractions around file uploads, and use that as a way to illustrate what are the fundamental pieces that are not specific to the, as you said, input APIs, and things like that, that actually do play out whether youāre doing this with bare platform, bare HTML and JavaScript versus something a little higher-level?
Yeah, absolutely. So this conversation is kind of based on a blog post series that I write, and I donāt think that my writing is particularly unique or special. I think that thereās a lot of excellent writers out thereā¦ But I think a gap that I try to fill is in all of these blog posts that I read, that kind of miss either the next logical step, or just like go around some lower-level concept that is necessary, or you want to understand how this stuff actually worksā¦ So I try to provide this fundamental knowledge in what Iām writing. At least like the one layer below.
[22:01] So to get into this sort of topic, I think the place to start with file uploads for me - I generally lean more towards the frontend. You could start in different areas, but I always like to start on the frontend, because thatās what I see. And the most basic fundamental thing on the frontend is HTML. And even that, if weāre talking about file uploads, you canāt just talk about file uploads by understanding HTML, you have to understand the fundamental thing below it, which is HTTP. Because you need to make an HTTP request in order to upload a file. Yes? Cool? There so far?
Youāre now going to teach us HTTP. Go!
Yeah, no, Iām not going to teach you HTTP. I probably donāt understand it well enough.
And to understand that, you have to understand TCP/IP.
And to understand that, you have to understand Ethernet.
Thatās the level that I probably donāt understand.
[laughs] There you go. So we all have our limits, right?
Yeah. And again, you donāt need to understand everything all the way down, but you do need to understand the things kind of at the lower level, from the point that you work in.
Right below you.
Yes, the next level below at least. So with HTML, to send a file, you need to do a number of things. In HTML, the communication language between computers on the web is HTTP, and to send a file with HTTP, you need access to the file contents. You need to modify the request header to send a post method, and you need to set the content type to multi-part form data.
So if weāre at the level of HTTP, can you break down - what is the difference between a get and a post, at least? Because I think that is actually a key thing and something that we sometimes paper over with tools like GraphQL, and start using in strange ways.
Yes, thatās a good point. I think that thereās going to be probably some areas of knowledge that yāall feel free to fill in. As I understand, an HTTP request is basically a protocol that you constructā¦ I want to say it kind of looks like a string of text that a computer can send to another computer, but the way that string is constructed is important. So you define the HTTP headers, the version, and then the method, and the method can be I think one of either seven or nine different options. But with just HTML, HTML really only has access to two methods - the get request and the post request. And the get request is generally saying, āIām trying to receive some dataā, or āIām asking for data from the other computerā, and a post request is saying āI have some data that I want to send to youā, to another computer.
Yeah. So the biggest difference with that, falling within those params is that a post can also have a body on it, whereas get doesnāt.
Yes, it is a ā I donāt know if itās against protocol to include a body on a get request.
It would often be ignored.
The only way you can pass data, or get requests ā well, there are two ways. Thereās the headers that youāre sending, and then thereās params, which are part of the location, the URL string that you are sending this to. So any HTTP request youāre sending is going to a place, conceptually, a virtual place, but that is a URL, and it has a set of data passed as params, which some backends will treat as giving you more information about what place youāre looking for. Like old school WordPress, youād pass a parameter and that would tell you what page youāre looking for, things like thatā¦ Others will just treat it as essentially modifying, but thatās the only way you can pass data along. And the headers are relatively constrained, the params are not. You can put whatever you want in there.
But you could also pass some data ā a REST API might pass some data via the URL itself. So you have the URL location, and then you also have the query string at the end.
Yeah, that location can be interpreted as data by the server. But you canāt, for example, put a blob of binary data or something else in that.
But with a post body, you can.
You can do that. Yeah, and then you also need to set some content type along as the HTTP headers to denote what type of content youāre sending. So this is really a fun conversation, because now weāre actually getting into practical knowledge, that I think ā
Letās just start the show right here.
Yeah, exactly. [laughter]
Right, right, so thatās a problem. Weāve got to Figure out a basketball, or soccer, or ā
Some sort of way to derail Austin before he gets into detailsā¦
Can we do field hockey?
Okay, so field hockeyā¦ So whatās ā
This is considered a pivot, I think, but in basketball if you lift that pivot foot, itād be traveling, and ā
Thatās right.
And Nickās exhausted his knowledge right there. Then what happens, Nick? [laughter]
I have the Wikipedia page open for basketball right nowā¦
Okay, but the ā thatās brilliant. Kball, Iām glad that you took us down that, because I didnāt touch on that lower-level HTTP protocolā¦ Which is funny, because itās redundant to say HTTP protocol, but I do it every time.
Thatās right.
Because itās the last P, isnāt it?
It is the last P.
The last P stands for protocol.
Hypertext Transfer Protocolā¦ Yeah. Who cares? Okay, so weāve kind of established that fundamental knowledge of HTTP, right? And then you get into the browser and the language of the browser, the sort of base-level language of the browser is HTML. And with HTML, you have access to making get requests using links, you can make get requests using formsā¦ But thereās only one way to send post requests, which weāve established is necessary to send a file; the only way to make a post request is with an HTML form. And we also established that you need to have access to the actual fileās contents, and you have to send the multipart form data HTTP header, right?
So with the browser - the browser provides a sort of sandbox ā itās an application. Itās a computer application that communicates, and it translates HTML into a visual layout of a website, and it constructs HTTP requests for you, without you necessarily having to know whatās going on. But the browser sort of works as a sandbox environment that doesnāt give access to website developers, doesnāt give them access to user file systems. Like, you canāt build a website that accesses its userās file system.
A little asterisk there, because there are some experimental technologies about file system access, and thatās sort of an aside, because itās a limited file system accessā¦ So before anyone kind of tramples on me, letās just say ā
I was so close.
I know. You donāt have access to like the full file system. Any file in the userās computer, right? In order to do that, you have to have user interaction where the user can select the file, and in order to do that with HTML, you can only use an input with a type of file, right? So now we have access to the file, we have a form to construct a post request, you set the method on the form to post, you have to set the content type to multipart form data, which you do with the [unintelligible 00:28:42.17] type attribute on the form elementā¦ And bingo-bango, you can send files with HTML. Thatās kind of the bare minimum. Cool?
And so what thatās doing under the covers is that file as a binary blob is getting sent along as the post body in HTTP. So then anything thatās responding to that, something on a server somewhere, is able to access that blob.
So itās also important to point out that if you leave off the method, browsers default to using a get request. So it just adds data from the form in query string parameters. If you leave off the [unintelligible 00:29:21.16] type, then browsers will send the body of that file, or itāll append the data from that file either in a get request to the query parameters, or in the body as just a string of the file name, I believeā¦ So if you want to have access to the actual binary data from the file contents, it has to be multi-part form data. And it doesnāt actually send it as a one blob in the request, because a file can be a large thing, so it can exceed the amount of data that can be included in a single request. So what the browser will do is send that as a stream of chunks of binary data. So like send little bits at a time.
[30:06] Then the next question that people ask is, āWell, yeah, HTML is cool. Iām glad I can do thatā, but everyone wants to do the thing where the page doesnāt refresh, because when you just use HTML, you hit submit the form, the formās gonna reload the page, and whatever. We want to provide a nicer experience for users using JavaScript. So then whatās the next question with JavaScript? Well, you still have to do the same things. You technically donāt need an HTML form, because now you can construct an HTTP request using XML HTTP request, or fetch. Most likely fetch. So fetch can still construct the HTTP request, provide the URL, set the method to post, set the correct HTTP headers, and then you still need access to the file system, which still requires at least a file input, because you still need user interaction to select the file, because the browser doesnāt have access to the file system.
Thatās worth actually ā I think thatās a fundamental thatās worth diving into a little bit more as well, which isā¦ Thereās a set of things that you can trigger on your own with JavaScript. And thereās a set of things that require user interaction. And thereās, I think - I donāt know off the top of my head what the list of things are, but thereās a set of things where basically you have to beā¦ What that plays out in is you have to be ā well, either itās handled by the browser, like attaching a file via an input, or you have to be inside of a user-generated event handlerā¦ An event handler for a user-generated event for it to work at all. Do you happen to know offhand what is that list of things? Or maybe even better, whatās the conceptual model behind when thatās going to be true?
Dang, you put me on blast.
On the spot. [laughs] No, no, no, I could not tell you. I probably couldnāt even get close. But yeah, there are some things that can only happen within user-generated events.
Most of the time once you try to do it, itās logical. Youāre like āOh, this makes sense.ā And itās usually security and privacy. Youāre like āWell, Iām not going to just be able to send myself arbitrary files, because you loaded my webpage.ā But once you interact, you can select the files, and now I have access to that thing. That just makes logical sense with regards to privacy and security and user intent.
If you want to know all the things you can do without any interaction, visit themostannoyingwebsite.comā¦ Ferossā website that will trigger every crazy thing that JavaScript can do, all at once, and youāll have to actually probably reboot your browserā¦ So donāt actually visit that, but if you want to, give it a shot.
You should do it while weāre recording, Jerod.
Yes, I should do it right now, and then Iāll just disappear.
Yeah, yeah. Shout-out to Feross, actually, because Kball, thatās a really good point that touches on, again, fundamentals that really lean into security principles. Because security is one of the pillars of web development, or requisite knowledge, and understanding what can happen ā like, thereās a lot of dark patterns around tricking the browser to believe that a user did something, or attaching events to certain user interactions to then do things that would not otherwise be intended, like maybe clicking on hidden or a not visible thingā¦ I donāt know. We donāt have to dive into it, but Iām glad that you brought that up. Again, fundamental knowledge that apparently I need to fill in.
I think it is key to realize, browser vendors are basically trying to protect their users, and make sure that you are doing things ā when you do things to them, it is because they asked you to. And so anything that is going to involve touching something from their computer, doing something that feels intrusive, launching a pop up, all these different things - browsers will try to make sure that youāre doing this in response to a user event. And that has been an evolution over time, as people like Feross found all of the edges and the places where you could do things to a user that they maybe didnāt wantā¦
[34:03] And browsers have evolved over time. I know weāve had conversations with folks who used to be in ad tech, where they were like āIt was an arms race between us in the browsers, where we kept trying to figure out how can we do a pop over, a pop under, a be more in their face, and the browserās kept being like āNope, nope, nope.ā But that core concept of fundamentally the browser is attempting to defend the users from malicious programmers is, I think, a key way to think about what it does and does not enable, and how it enables things when it enables them.
And I donāt think that this is a was thing. Itās still actively ongoingā¦
ā¦and itās personified perfectly, I think, in the recent Apple Vision Pro announcement, where that has a Safari browser on it, but it is explicitly going to protect the user from where their eyes are gazingā¦ Because it can know exactly what youāre looking at in Safari, but itās not going to give that information to the user, because thatās so much more private than where your mouse is, for example.
Wait, stop. Tell me more. How does that - like, Apple, Safari in there wonāt give you access to it?
To user eye-tracking.
But isnāt that how a lot of things are triggered?
Itās at an OS level which wonāt leave the device.
So web developers building websites in Safari on the device wonāt have access to eye tracking technology. However, the operating system does.
Yeah, so theyāll get access to higher level events, like āOh, this is a synthetic click eventā, or whatever, that may have been triggered by eye-tracking, and I was looking at it, and whatever, but the underlying eye location data is not going to show.
Itās all about consent.
Yeah. So we consent to you continuing your tutorial.
Awesome. So weāve essentially sent things from the client side. And I think outside of the GraphQL stuff - because thatās outside of the scope of my knowledge, uploading files with GraphQL; I donāt know - probably at some level itās HTTP, so youād have to do the same thing. Thatās the lower-level ā
I mean, everything in GraphQL is a post, so itās in some ways a shorter gapā¦
Yeah, yeah.
Though having the combination handler that understands both GraphQL and the file - thatās anotherā¦ A special snowflake.
Yeah. Oh, GraphQL error handlingā¦ Itās awesome.
Everythingās 200. Itās good.
Everythingās a 200. Great.
Alright. So we made that reference. Weāre talking about fundamentalsā¦ Whatās the 200?
Oh, goshā¦ [laughs] A 200 is a status code that says everything is ā I think itās a status code for receivedā¦
Okay, I think.
Itās okay, isnāt it?
Or okay? Yeah, that the request went okay.
āYou sent me a thing, I got it. Itās all good.ā
Even if itās not good, becauseā¦ GraphQL.
Yeah, actually that was a ā the status codes is a sort of a milestone in my career, where I recognized that I understood the fundamentals of HTTP better thanā¦ You know, I started as a frontend developer, got into backend development, and then did a whole bunch of mistakes on how to handle communication between the two, because I was trying to do things at a higher level, and send things in like JSON responses of telling the client if the request was okay or not. And then you realize that no, if you just follow these decades-old protocols that all of this is built on top of, and you kind of opt into those, you now are doing things using tools that have been built over decades, and have a much larger ecosystem, therefore a much larger knowledge base; youāre not building bespoke things that only apply in your case, and everyone else that joins your organization is going to do things in a standard wayā¦ And then when you move on, you can take those standards with you. Itās great.
Well, and that points back to your original point of the value of fundamentals, right? You donāt end up reinventing the wheel. You donāt have to redo all of these things, and thereās a lot of stuff that you get for free at the platform level, if you follow platform-defined semantics.
[38:01] Yeah. Okay, so we sent out files from the frontend, right? Now, the lot of tutorials will stop thereā¦ Not this guy. We want to move on to the backend, because thatās the other side. Like, anyone can just send files anywhere, you know? But I want to receive them. So this is a JavaScript podcast, Iām a JavaScript developer, weāre building applications in Nodeā¦ You might use Deno, or Bun, or something, or one of the other edge runnersā¦ I donāt know. But anyway, a lot of these platforms are adopting Node standards, which is great. We want portable code, thatās awesome. And the way to receive file uploads - again, touches on HTTP; you know that youāre gonna be receiving a post request, you know that that header, the content type header is going to be multi-part form data, so you need to tap into that. And again, because we discussed how the browser is sending data one thing at a time, you have to be aware of that data coming in at different events. So in Node.js, a lot of people are familiar with Express, but even at the lower level, Node gives you access to an event handler system with the incoming message request object. So you can say request.on, and then listen to the data event, and that data is going to come in one little chunk at a time from what the browser is sending. Itās not all going to be in one massive payload. Because if weāre dealing with a 10-gigabyte file, too much for one HTTP request, right? Or one chunk.
So that becomes interesting, because then you have to do something with that data. And the data comes in as chunks of binary code, which JavaScript doesnāt really know how to deal withā¦ And so JavaScript has this concept of buffers, which are also confusing, because when you log into the console, itās just something that says, āGreater than buffer, a bunch of string ā of numbersā, and then close bracket, right? And youāre like āWhat the heck is this?ā So you can convert it to a string, then you can start to see the little bits of string representation of that data, and thatās niceā¦ But you still have to do something once it all comes in. So the naive approach is to say āTake this chunk of data, convert it to a string, push it into an arrayā, and then as each chunk comes in, you push all of those things onto an array. And thereās a problem there, because eventually, that array is going to take up more and more space in memory, which is fine for small HTTP requests, but because files can be quite large - letās say a 10-gigabyte file, if youāre running on a server that has two gigabytes of memory, itās going to exhaust those resources. And everyone thatās had to create file upload scenarios probably has run into the server crashing, because it ran out of resources.
Well, and this is highlighting another fundamental concept, which is streaming data, versus serialized data.
Exactly. And the streaming stuff - for anyone listening, if this is your first experience with streams, itās really confusing. I donāt even want to try and tackle explaining it, because it took me probably like 10 times of reading the same thing, reading different things, doing reading, versus video, versus presentations, before I actually had a decent grasp. So I do want to stop and say, if you are coming into streams your first time, be kind to yourself, and understand that a lot of people have problems with it, and thatās okay. Youāll get around to it.
I mean, I think there is a fundamental sportsball metaphor that we can make hereā¦
Sure, letās do it.
Letās do it. Sportsballā¦! So imagine that you have three people in spread out from each other, and you have a pile of soccer balls. You have like 10 soccer balls, and they start with one person. And you want to pass those balls along your lines, through a fireman style, one, two, until they get to the end. A stream would involveā¦ A person starts with ā has 10 balls, they kick one to the middle person, that middle person immediately kicks the ball forward. And so as you transfer these balls, the person in the middle, as soon as they get a ball, theyāre kicking it forward. They donāt have to have a lot of balls all at once.
[42:03] A buffered approach, or a serialized approach would mean you kick all the balls to that middle person, they collect them all, until they have their 10 balls. Then they turn around and they kick those 10 balls off to the next person. And so you end up with everything together at each step, bunched along. So bringing this back to programming, streaming is essentially saying, āHey, Iām only dealing with a small chunk of data at a time, and I do whatever processing I need to do on it, and then I pass it on and I forget about it. Iām not worrying about it.ā And that is fundamentally more memory-efficient, because youāre only storing in-memory that small amount at a time.
Now, thereās things that are harder to do like that, right? Like, if I wanted to analyze an input file, some things I might be able to do easily. Say Iām counting the number of aās in a string - okay, I keep one counter. Look at this chunk; do I have any Aās? I add them on, I pass it along, whatever. But if Iām doing like a whole image analysis, thatās not going to work as easily in a stream-based manner. So there are problems that are easier to do in a streaming way that not, but fundamentally it has to do with āAm I storing up all the data as itās coming in, so I can look at all of it at once? Or am I looking at each piece and then passing it off and forgetting about it?ā
Yeah, thatās a good point. I work at Akamai and we have edge compute technology, and one of the things that you might want to do with edge is you receive HTML from the origin server, and then you might want to do some sort of rewriting of the HTML, and then send it to the clientā¦ But because youāre dealing with chunks of data at a time, what happens if you want to modify, letās say, the body tag, and one chunk of data comes in with open bracket BO, and then thatās it, and then the next chunk has DY, close bracketā¦ You canāt ā itās really hard to tackle string replacement across chunks.
But anyway, so going back to the streaming thing, like, Kball, you were saying - that does introduce streams, right? You want to avoid writing files and potentially large payloads into memory because of that. So streams are the solution, where you open up a file write stream, and on the server thatās running your application, as you receive each chunk of data, you maybe convert it to a string, and then you write it to the file system, and then you release that memory. And then the next chunk of data can come in, or multiple chunks can come in at a time, but youāre writing them to disk and then releasing them.
Now, Kball, this concept comes back to one of the points that you made about abstractions. Weāre talking about the very low-level Node APIs that are available. And at this point, weāre talking about ā weāve basically described creating your own server, HTTP server, and creating your own file or routing system and file handler. You donāt need to do that. Weāve had Express forever, Fastify is awesome, and they provide these higher-level tools. That makes sense, because I think you should understand how Node does thisā¦ But at this point, these are tried and true, and Iām not going to improve upon them. And they provide all the tools that I do need, and that I will need. Itās worth it to opt into those.
Totally. Well, and the concept of streaming data is transferable, right? You can write streaming versus batched operations in Python, you can do it in Go, you can do it in Rubyā¦ That core concept is going to be transferable regardless of whether youāre learning it first in Node, first in Node with an abstraction, or what have you. Wrapping your head around that distinction is where the fundamental transferable knowledge is.
Yeah. So those abstractions are good to know. In the article that I wrote, I go into ā basically, Iām building this application in Nuxt.js, which is a very high-level abstraction. Nuxt.js is a metaframework of Vueā¦ But because I know Iām going to deploy it to a server thatās running Node.js, I like the fact that Nuxt.js gives me an escape hatch to tap into the lower-level system of Node. And once I have access to Node, then I can write a middleware, or do things in the Node way, that I can take that chunk of code and bring it in and use it anywhere.
[46:06] So I can write a file stream handler using the Node ecosystem that will work in Next, in Nuxt, in just plain Node, whatever. And that also gives me access to the Node.js ecosystem, which is abundant, with tools like file system stream writers like Formidable.
So a library that I really like thatās part of the Node ecosystem is Formidable. It was written a long time ago, where everything was done with the error handling upfront, the callbacks approach, where you get the error message, you check if thereās an error, whateverā¦ And so you can take that and you can write essentially a function that passes the underlying incoming requests from Node to Formidable, and then you have your own little well-tested, established tooling that can then live and be put into the Nuxt ecosystem, because you have a good understanding of where those boundaries are of what is the Nuxt responsibility and available tooling, what is the Node responsibility, what is the Formidable responsibility, and how do they interact with each other and interface with each other. Yeah?
You guys are nodding your heads, but this is an audio show. Please confirm audibly, yes. Iām here to say yes. Nick?
Thank you. Kball?
Iām already talking enough. Yāall can handle this.
[laughs] No, I really appreciate that. Kball, youāve been doing a great job of checking me, so I appreciate it.
So yes, I think thereās something more than just understanding the lines, which is that a well-designed framework or metaframework will actually give you seams where you can insert things from lower-level abstractions. They will make it easy to do. I think this is something ā if anybody out there remembers the Ruby on Rails ecosystem from 10 years ago, Rails was bad at this when it started. It was āWeāre gonna make a choice for you, and itās hard for you to sub in your own choices.ā And over time got much, much better and cleaner, of saying āOkay, hereās how you sub in your own choice. You can alternate āā, and it kind of created these seams where thereās like a little thin abstraction layer around an underlying piece, or technology.
So I think in the JavaScript ecosystem right now, when you look at something like Nuxt, theyāre doing a reasonably good job of that. Theyāre like āOkay, youāre running something on the server; hereās where that goes, and within that, you can plug in anything from Node you want.ā And so theyāve made it relatively straightforward to follow your understanding of fundamentals, and like cut in the different pieces that you want, where you want it. I think that that is something to look for also when youāre adopting these higher-level abstraction, is āDo they provide those seams, those cutpoints for you to come in and inject changes that operate at a different level of abstraction?ā
Which is hard to do.
Because if they donāt, thatās where you paint yourself into a corner, and then all of a sudden āI need to do something that they donāt offer, and I donāt have access to the underlying ā the thing that theyāve abstracted over, and now Iām stuck. Or I have to find something else.ā Thatās the worst.
Okay, so I think that that pretty much covers the technical side of things. But I donāt think that that paints the whole story of what to deal with and what to think about when youāre allowing for file uploads, because - well, first off, it doesnāt look at the holistic architecture view of building applications these days, which could be distributedā¦ So if you are building a distributed system, and you are receiving files and writing them to disk, but you have multiple computers that are processing your requests, all of a sudden itās really hard to track down where those files are being stored. Which server, which location, and stuff like that.
Which is why you stream them to S3 or something equivalent, right?
[50:02] Exactly. On top of that, on top of the trouble with distributed systems is what happens when you start receiving large files and you have a server thatās maybe like a lightweight server, and totally enough for your application to run, but it has smaller disk space, and all of a sudden, large files are taking up a lot of the disk space, you run out of disk space, you have to upgrade your server. Now, if you have to upgrade your server because you want to add more compute, youāre getting sluggish, or you need more CPU - excellent reasons to upgrade a server. If you are running out of disk space, terrible reason to upgrade a server, because you may end up doubling the cost at a not very cost-effective way of essentially doubling the disk space. And before you know it, youāre running out of disk space again.
So Kball, you mentioned S3ā¦ S3 stands for Simple Storage Solution, I think. S3 was created by Amazon, along with their service. They created their own protocol for storing files, adding files, managing files, deleting, updating, whatever. And itās been great. And that protocol has been adopted by a lot of other file storage solutions partners. I do work at Akamai, I feel like I should interject that we have an object storage service that we offer, that is S3-compatibleā¦ And the article that I write, again, speaks to fundamentals. Yes, I work for a company that offers object storage solutions, yes, in my examples I use thatā¦ But everything that I write, I try to do, again, at a fundamental level, so you can take this and apply it to anything that is S3-compatible. So anything that uses this protocol, take it and put that knowledge where you want it. Very important to me.
But one of the big factors for object storage, S3-compatible object storage, is that they are designed for these files. So theyāre highly available, easy to access, they store it in one location, and theyāre very, very cost-effective. So for the same price that you would get 25 gigabytes more on a VPS to upgrade your VPS, you can get 250 gigabytes, at least with Akamaiās cloud computing services. So thatās a 10x difference. And that applies ā and again, that would be across different service providers. Iām only saying the numbers that I know off the top of my head, butā¦ Not trying to turn this into a sales pitch.
So in this case, the core concept is object storage, which ā
Yes, object storage, whoever you go.
ā¦is essentially like the distributed version of file storage, right?
Itās like, āHereās a location, that leads to a file, that has some set of permissions.ā And you can even do directory structures the same way you might in a file systemā¦ But is there anything more to it? Is there any other concept? Or is it ā if somebody is used to thinking about files, is this just files in the cloud?
I mean, I would say basically yeah. If youāre looking at object storage solution providers, I would encourage looking for S3-compatible object storage. I think most object storage providers are S3-compatible, because itās become such an industry standard. And the benefit there, again, is you have portability. You can write your code in one way, and as soon as a different provider, a different S3 provider, or S3-compatible provider offers something that is a deal-breaker for you, or maybe the current provider that youāre using changes pricing, or planning, or how youāre allowed to access your data, and you want to transfer it - cool; you donāt have to change your business logic of your application. Thatās really important to me.
Any more on object storage? Itās highly cost-effective, regardless who you use, and also very simple. And the key is - I wonāt get into the weeds on the implementation details, but the interesting thing when we bring it back to how you move your files from writing to your server, your application server, to object storage, matters, because thereās a couple of different ways. And the question is how do you get those files into the object storage provider? And I think almost every piece of content that Iāve found out there suggested using whatās called signed URLs, where essentially your client makes a request to your backend, the backend serves as a proxy for the object storage hosting, and the backend will make a request to the object storage provider for a signed URL. A signed URL is essentially the location where the file is going to be uploaded to, but it can only be uploaded to by the person with that signed URL. Itās basically a URL that includes some sort of authentication mechanism, or authorization mechanism, as well as some expiration date.
[54:41] So your backend can provide that to the frontend, the frontend can take that, then upload the file directly to the object storage provider. It doesnāt have to go through your backend at that point. And they can get the file there. Thatās what most people say is the best way, and it is a decent way. I personally prefer building my applications with progressive enhancement built in. You cannot do that, I donāt believe, in a progressive enhanced way, because I believe the only way to use a signed URL is sending a put request, which HTML cannot support. So that breaks. But there is a different way, and thereās a couple ā Iāll explain the other way first, and then maybe we can look at the pros and consā¦
The other way is you can send the file directly to your proxy service. And again, instead of streaming to writing the file to disk, you can pass that stream along and upload it as a stream to the S3 storage location. The big downside there is that - well, thereās a couple of downsides. One, you are basically paying potentially double for that file, because you are going to pay For the file being stored in your object storage, but you may also be paying as your server, which serves as a proxy, is receiving that file - thatās ingress cost; and as it streams that file out to the S3 provider, that is egress cost. So because itās passing through this proxy, you may be paying more, because of the traffic thatās being run through. Your mileage may vary, check with your stackā¦ Again, you can take this to anyone.
The other sort of advantage or disadvantage is because that file is passing through your proxy servers, now you have more control over what to do with that file. For example, if you wanted to do file optimization, if itās images, you canāt do that if people are uploading directly to your object storage provider, unless you also pull it down, in which case youāre going through your server anyway. The other sort of downside is, once again, because that file is passing through your proxy server, what happens if someone wants to upload malware? Now, if they upload malware through a signed URL, itās going to go to the object storage. Itās kind of in quarantine there as long as you do some sort of malware scanning on itā¦ But if youāre passing a file through a proxy service, that malware could potentially be into your system, and then pass through. So then the question becomes, āHow do I deal with malware on my uploads?ā And once again, I think that this is the advantage ā as much as it might be a disadvantage, itās also an advantage of passing files through your service, because it gives you an opportunity to actually scan those files for malware. Any questions here so far before I continue into malware?
Iām waiting for the malware apart, man. Well, how are you scanning these things?
Okay. Okay, so dealing with malware is interestingā¦ OWASP has a whole cheatsheet that I can link to, or is in the blog post that I link fromā¦ OWASP has a whole cheat sheet on how you deal with malware in the context of file uploads, and itās pretty interesting, because once again, this sort of library, Formidable, that I had mentioned before - itās really awesome for dealing with files, because it handles all of this stuff that you would want to do sort of for you, or at least provides an easy API interface for doing thatā¦
One of the things is you want to ensure that ā well, essentially, to deal with files or malware, you kind of have to have the entire content of the file. So remember how we said you could stream a file from the client to your proxy server, or to your backend, and then pass that stream along to the object storage location? If you want to address malware, you may not want to do that; you may want to go back to writing it to the disk on that server, because you need the entire contents of a file to know whether itās malware, or to scan it for malware.
[58:23] Right. But you could just stream it to like temp storage or something, and thenā¦
You could stream it to temp storage, or what you could do ā yeah, and you could have a temporary sort of quarantine folder on disk, that is the temporary storage, and then pass that to object storage again, which is probably how you would do this if you went this approach.
Or have like another bucket thatās like your temporary object storage bucket, if youāre thinking distributed systems. Those disks ā I mean, local disks just destroy distributed systems when it comes time to actually distribute themā¦
Yeahā¦ I mean, yeah, you could have another object storage location, but if you want to scan it, youād still have to pull it down to some server, and then scan it thereā¦ So it would kind of be redundant. So I think what you might do is
Well, it depends on how long these processes take, right? You could throw it in a background queueā¦
Thatās true.
ā¦and you could have a bunch of workers that are working off the backend to just scan those thingsā¦
Thatās true. You could have you could have a separate backend just for scanning files.
But I donāt know how long it takes. That maybe helps inform this discussion.
Thatās a good question. So a lot of the OWASP recommendations are like obscuring file names, or not using the file names from the client, because that could use disallowed characters, checking file types, checking content, and the [unintelligible 00:59:38.03] or the MIME type of the file, and sort of like filtering that way, preventing certain file sizesā¦ So if you have a system thatās like only scanning PDFs, you donāt need gigabytes of dataā¦ Versus an application that is going to be doing like video editing - you may need to support multiple gigabytes. So this is where you get into like more bespoke decisions around what your application is.
But ultimately, you can have malware scanners running. Like, basically receive the file, write it to disk, run a scan on it, but that can take a long time, and therefore you canāt really build a system that does that within the request/response cycle, because a request and a response has to be quick. Basically, just the time it takes to upload it, and then I need to know that itās there. So you would want to build some sort of background process that then requires a database, that keeps track of all of the metadata about the file: the file name, where it exists on disk, what the MIME type is, and then some sort of other information of whether it has been scanned or not, and then youād have some sort of background process that says, āOkay, check the database, look for the files that have not been scanned, pull them off of their location, download them, scan them if theyāre good, put them back, mark them as greenā¦ā And then this kind of comes back to āAre you going to be presenting these files back to a client somewhere?ā Because if you are, you want to be able to upload those files, you want to be able to tell people that their files are uploaded, and you have a record of them; so you might show that to people on the frontend. But if they have not been scanned, you want to show them the metadata, but not provide a link to download that, because you donāt want to ā even if you protect your system, even if malware gets into your system and you have a way of quarantying it ā quarantining itā¦ [laughs]
Corinthiansā¦?
Something in thereā¦ You know what Iām saying; protecting it.
Are we doing Bible verses?
Yeah. [laughter]
1 Corinthiansā¦
You want to be able to protect your system, because you donāt want to ā you donāt want to give a file access to maybe reading your database credentials, or whatever. But you also want to ensure that your system can be trusted by your users. So you donāt want to become a source of distributing malware.
[01:01:50.21] Well, and you can see this example anytime you upload a file that is large enough that itāll take substantial time to a social network, or YouTube, or something like that. Theyāll say, āOkay, this is uploaded. We are scanning it.ā Then once itās scanned, itāll go live. And the big guys make it real fast. They do what they can to make that happen really quickly. But it can take some time.
I think to Jerodās earlier point, by the time youāre doing all this tracking and background tasks, itās going to be simpler if you build distributed-first. So you put it someplace in S3, you track it, then you have whatever your background cue is; theyāre the ones pulling it down locally, and they donāt have the latency worries. So they can just pull it down locally, do all the scanning that needs, mark it off wherever it is; you donāt have to do quite as much bookkeeping, in that sense. So yeah, I think once youāre thinking distributed systems, object storage all the way.
For sure. Hopefully ā I mean, I think that kind of tied up the whole solution. We did HTML, JavaScript, Node, metaframeworks, storage solutionsā¦ We didnāt talk about delivery, which is a CDN to basically make sure that files that are being downloaded to people in Tokyo, or Toledo, or Toronto - they all get them as fast as possible.
I should just mention that CDNs are basically like a globally distributed network of connected computers that can be used to automatically identify traffic, and theyāre sort of distributed as close to users as possible. So they can route traffic to the nearest node in the network from where the user is coming from. And theyāre useful for storing things like static assets, so that people donāt have to request things all the way from an origin or an object storage location. And they also these days are a lot more advanced; they can offer features like web application firewalls, DDoS protection, and other awesome security features.
So coming back to then your file example - after somebody has uploaded this file, theyāve stuck it in an object storage, CDN basically will ask that object storageā¦ The first time somebody requests that file, theyāll ask the object storage, theyāll get it back, and then itāll store a copy of it all out on these edge nodes, which are computers all over the world close to where users might beā¦ And then when future requests come in, following all the cache protocols, and whatever your cache timeouts are, itāll get it just from that local edge, not even go back to that object storage.
Kind of yes. I think most CDNs work with a poll feature instead of a push feature. So when you upload a new file, you donāt push it to the entire network. Itās just when a user that goes through a specific node, that node, the first time that request is made, will pull the file from its origin, which might be an object storage location, and then store a cached version. And then any other user that passes through the same node will get that same file stored and delivered at like a fraction of the time.
Got it. So itāll go back to the object ā you know, Akamai has thousands of nodes out there.
So I might get thousands of hits back to my original object storage, but only one for each location.
Yeah, so it could reduce some of the bandwidth needs for your origin server as well. Because if youāre gonna get thousands of hits from users around the world, youāre going to receive those the first hit, no matter what. But then if theyāre coming from the same location, you would only get one hit, and pay for thatā¦ In theory.
In theory. Everything in theoryā¦
I was struggling to find a sportsball reference for that. Itās like tennis, the guy standing at the net, and you hit the ball over, and heās like āNope.ā Hits it right back to you. But it doesnāt work quite right.
Yeah, I canāt ā I donāt have a good one. I watch roller derby, soā¦
Thatās alrightā¦ Another episode is in the can. And just like coaching kids in basketball, understanding JavaScript file uploads is just mastering these fundamentals. So whether itās a three-point shot, or a three-gigabyte file uploadā¦
ā¦practice makes perfect. So keep coding, and keep coaching, and weāll catch you next time.
Changelog
Our transcripts are open source on GitHub. Improvements are welcome. š
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK