How we update our Angular web app
source link: https://blog.shapeq.io/how-we-update-our-angular-web-app.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
How we update our Angular web app
Jul 21, 2022 • Matthijs van Niekerk
When developing a web app, sooner or later (usually sooner) you will want to update it. Doing this intraday, without disrupting our clients who are actively using our app, having no downtime during updates, and caching as much data as possible to minimize experienced latency requires a bit of setup.
In this post I’ll give an overview of the steps we have taken to make our frontend updating process as smooth as possible, in the hopes that our process may prove useful for others. Setting all of this up was not a one-day process, but rather a number of incremental improvements stretched out over months.
Objectives
- Roll out updates quickly to all clients. Since our app is a web app, this should be as easy as pressing the refresh button. Main challenges here are making sure that the refresh loads the updated version of the app (and not an old cached version), and that we ask the user nicely to refresh, causing as little disruption in the process as possible.
- There should be zero downtime during updates. Our backend should only stop serving our old frontend files after the new frontend files are ready. Once the files for the new frontend are ready, we do not need to keep the old version around.
- Cache as much of the app as possible, to minimize load times. While we are taking active steps to reduce the footprint of our app, this process is not finished. The total bundle that a user has to download when they use our app still clocks in at 4MB uncompressed or 1MB gzipped.
The user perspective
When we update the frontend to a new version the user sees something like this:
The bottom right of the page shows the popup. The only way to get rid of the popup is to click it. This will refresh the page, loading the new version of the application.
There is a tradeoff here. On the one hand we want the user to update as quickly as possible. After a major update the backend api may have changed in a backwards incompatible way. If the user keeps using the old version of the app, some actions that the user wants to do may error. On the other hand we do not want the user to loose work. If they are in the middle of editing a form or sending a message, we want to allow them some time to finish and send it out before we force them to update their app.
Tech stack
The frontend of our app is built with Angular. The backend consists of a number of services written in either node or python, running in docker containers orchestrated with kubernetes. The frontend is served through containerized nginx on kubernetes.
Angular
Angular uses a single-page application setup. This has a number of implications for updating.
Starting from the moment that a user launches our app, there are no full page reloads. When a user moves through the app different views are loaded in dynamically.
When building an Angular app for production, the dist (build output) directory will look similar to this:
Note that all .js and the styles.css file have a random hex-string appended to their filename. This is because the output-hashing flag is set to all in the angular.json file. The hex-string is a hash of the file contents. This means that if the file content changes after an update, the file will have a different hash and therefore a different hex-string appended to the filename.
index.html in the build output is very small, and it will be the default page that is loaded in our app, regardless of the URL path that the user starts ther session from. Below is the complete index.html file of the build with the output formatted.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Test Project</title>
<base href="/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/x-icon" href="favicon.ico">
<link rel="stylesheet" href="styles.f8e3132b35359c87.css" media="print" onload="this.media='all'">
<noscript>
<link rel="stylesheet" href="styles.f8e3132b35359c87.css">
</noscript>
</head>
<body>
<app-root></app-root>
<script src="runtime.3842f07dbbb3f24b.js" type="module"></script>
<script src="polyfills.90cc9c800b15c100.js" type="module"></script>
<script src="main.a9f36f023616e3ad.js" type="module"></script>
</body>
</html>
There is hardly any HTML, and all the actual content is loaded in through the .js files. Because of the caching, the name of the .js files will change after each file change. Since the index.html file is the entrypoint of our application, it has complete control over what .js files to load. Because of these points, it also has complete control over what version of the application to load.
We can use knowledge about Angular to create a solid caching strategy. In our tech stack we do not use a reverse proxy. Because of this, the only caching layer we are concerned with is the browser cache. If we would have had a reverse proxy, the same logic would apply.
The index.html file is only loaded when a user loads the app for the first time. This will only happen once per session and the file is small, so it is no big deal to completely disallow caching for index.html. The index.html file will point the browser to the right version of all other (much heavier) .js files.
All files with hashes should get a different hash after each change. Because of this we can set a very aggresive caching strategy for these files.
All other files (like the favicon.ico and other assets) do not get a hash. However, they should not change too often, and we can let browsers cache them for a moderate amount of time.
Let’s see how these rules play out in practice. Using these steps for caching means that a user can encounter 3 scenarios:
- The user loads our app for the first time or loads it after an update. All files are retrieved from our server and hashed files are cached in the browser.
- The user loads our app for any consecutive time.
index.htmlis retrieved from the server and all other files are found in the browser cache. - The user is actively using the app during an update. They are still on an old version of the code, while in the meantime the files have changed. If they load a view that is part of a
.jsfile that is not in the cache, the resource cannot be fetched from the server (since the hash changed after the update) and the load fails. This is why we need to guide the user to reload the page directly after an update.
NGINX
We can configure nginx to follow the caching rules in the previous section. Translating the rules in the previous section to nginx configuration will look like this (only showing relevant parts):
# This block has lower priority.
# Should only be activated for actual routes (like /, /market/grid, /home/dashboard).
# Loads the file with all caching disabled.
location / {
# Try finding the uri in the files. If it fails, load index.html
try_files $uri $uri/ /index.html;
expires off;
add_header Cache-Control "no-cache";
}
# This block has higher priority (matches any .js or .css file).
# Hashed output files can be cached forever.
location ~* \.(js|css)$ {
expires max;
add_header Cache-Control "public";
}
Angular requires all routes to fallback to index.html if they do not exist. This complicates the nginx configuration somewhat because we cannot just have 2 simple rules that cache everything except index.html, since any route that does not exist in the actual filesystem should fallback to index.html. The configuration line that sets up the fallback is:
try_files $uri $uri/ /index.html;
The workaround is to cache files with a .js or .css extension, and run the fallback with caching disabled for all other files. The block for .js and .css files has higher priority since regex location directives have priority over path location directives.
Note that the configuration snippet above is not flawless. For example,
favicon.icowill now be matched by the/location directive and will not be cached. Also, most real Angular apps will also require to load resources from a resource folder, which would also require a separate rule.
Docker
NGINX is run from a docker container, with the build output files loaded as part of the container files.
A possible alternative would be to have a persistent nginx server running and load the build output files as a volume. With our current build automation (which expects docker containers as artifacts after each build) this is more difficult to set up, so for now we stick with one docker container containing both the nginx runtime and the build output files.
The dockerfile that manages this looks like:
# Stage 1: Build
FROM node:16 as node
WORKDIR /app
COPY . .
RUN npm ci
RUN npm run build
# Stage 2: Serve using nginx
FROM nginx:alpine
COPY nginx.conf /etc/nginx/nginx.conf
COPY --from=node /app/dist /usr/share/nginx/html/fe
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
Kubernetes
Kubernetes is responsible for orchestrating and running the docker container. Since the nginx docker container starts quite quickly, the default readiness probe (which just checks whether the main process has started) is good enough. After the new pod comes online, the old pod is downscaled.
This is also the point where we can quite easily track the status of our deployment. The package.json version is stored as the docker image tag by the build automation. The version is also stored as part of the build output files that are served over nginx.
Storing the package.json version in the build artifact is not exactly trivial but can be achieved using a helper script invoked before each build:
npx [email protected] --es6 _version.js"
This uses genversion to create a file called _version.js which can then be imported in the code as
import {version} from "../_version";
Another way to make this work is by importing
package.jsondirectly likeimport packageJson from "../package.json". This works whenresolveJsonModuleandallowSyntheticDefaultImportsare set intsconfig.json. However this is not recommended because this will import the wholepackage.jsonand package it in your build. Exposing the completepackage.jsonmakes it easier for an attacker to find vulnerabilities because it exposes package versions and build steps. So best practice is to only expose the version.
Now for a bit of magic: There is a node service running in kubernetes that keeps track of the current versions of all deployments in kubernetes. Every time the version of a deployment changes this service will emit an update to the frontend over a persistent websocket connection. This is the way the frontend can keep track of the latest version of the server. If the kubernetes version of the frontend is not equal to the local version of the frontend, the notification urging the user to update is shown. The relevant snippet of the server code is:
// create a new Kubernetes object (wrapper around @kubernetes/client-node)
const client = new Kubernetes();
// watch all deployment changes (as a rxjs observable)
client.watch("/apis/apps/v1/namespaces/default/deployments")
.pipe(
// take the name and version of the k8s deployment metadata
map(({value}) => {
const name = value["metadata"]["labels"]["app.kubernetes.io/name"];
const version = value["metadata"]["labels"]["app.kubernetes.io/version"];
return {name, version};
}),
// turn the stream of updates into a stream of snapshots of shape
// { [name: string]: Version }
scan((acc: { [name: string]: string }, {version, name}) => ({...acc, [name]: version}), {}),
// wait until there have been no more changes for 5 seconds, then emit a result
debounceTime(5000)
)
// send the resulting json structure to all clients with key `versions`.
.subscribe(versions => this.socket.send("versions", versions));
The implementation shown here only shows the business logic and skips some of the implementation details of the libraries used. The most important libraries used here are the kubernetes-client, rxjs for observable logic, and a custom library we developed internally for real-time state synchronization over websockets built around socket.io.
What’s next
In the future I could see we move our app towards a Progressive Web App with Angular Service Workers. This would give us even more control over caching than is now possible using nginx and the Cache-Control header. It would also bring other benefits associated with Progressive Web Apps. However, this is a major change that is not at the top of our priorities.
All in all, this post is a relatively complete high-level overview of our frontend update procedure that should contain some interesting information for anyone attempting to build something similar.
In a future post I would like to take some time to delve into our backend service architecture and kubernetes deployment update flow. Another interesting topic that I skimmed over in this post is the reactive way we write our code, where instead of writing procedural logic we create a functional reactive pipeline using rxjs where data flows through. Our frontend-backend state synchronization library is also a part of that architectural mindset.
I hope this post gave a good overview of our web app update flow. I think it contains some ideas that can be applied to other setups with a slightly different tech stack as well. More updates about the technical architecture of our app will follow soon, hopefully on a more-or-less regular schedule.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK