AWS S3 * 2

My new deployment workflow using AWS SDK for Rust

Once, when my brain was smoother, I hosted a static site on AWS Simple Storage Service (S3). I didn’t understand routing or callbacks, but there I was hacking an Express server to output flat HTML files. In a callback. Which meant to actually output HTML I had to crawl my own site. But after that, I simply called yolo.js to go live.

I still host my personal site on S3, though I use modern web tools like Svelte and Rollup that do my thinking compiling for me. Who needs routing when you can have a single page application? Web development today is like Olay Regenerist for the koala brain.

Two recent circumstances, however, have put a wrinkle in my smooth-brained bliss:

1. I foresee a sudden drop in personal time for development as I transition out of unemployment, and
2. AWS announced the Developer Preview of the AWS SDK for Rust.

Come along as I revamp my deployment workflow for hosting static sites on AWS S3.

Routing with AWS Smooth Storage Service (S3)

Say I have a static site hosted from an S3 bucket example.com and I want to add a “route” to another static asset in that bucket, say example.com/app.

How can I navigate to this “route” if the asset in question is literally named app.html?

I couldn’t. So I had two choices: I either use the Web 2.0-looking address example.com/app.html or just upload the file without an extension. 🤯

… S3 doesn’t really have a folder structure, but rather has a flat structure of keys and values with lots of cleverness that enables it to simulate a hierarchical folder structure.

— Mark Gilbert’s Tech Blog

While uploading files sans file extension is deliciously sneaky, I could have solved my routing problem by uploading a text/html file to the example.com bucket, simply called app/index.html. S3 is equally happy to serve example.com/app in both cases.

Static sites with Svelte (S3 * 2)

So, why all the hate for specifically named HTML files (with unspecified file extensions)? If the route fits, follow it! … Right?

The first reason I decided against my file extension hack is that **licks lips** I use Svelte for web development. All my (definitely reusable) components get compiled into one big Javascript file that I embed in index.html. With a virtual subdirectory structure, I wouldn’t have to worry about intra-bucket name collisions.

Given the canonical file structure of Svelte projects …

project
│   package.json
│   README.md
│   rollup.config.js
│
└───public
│   │   global.css
│   │   index.html
│   │
│   └───build
│       │   bundle.css
│       │   bundle.js
│       │   bundle.js.map
│
└───src
    │   App.svelte
    │   main.js
    │   ...

… all I need to host my static site is everything in public. If I add a step to my Rollup build that outputs an array of file names, then I can consume that array when I’m ready to deploy. To this end, I wrote an ES module that would output the following:

found 5 files in './public'
[
    'build/bundle.css',
    'build/bundle.js',
    'build/bundle.js.map',
    'global.css',
    'index.html'
]

The module saves this array to project/out/public.json for later batches.

Deploying with (AWS SDK for) Rust

The second reason I decided to go with a virtual subdirectory structure in my new workflow became clear as I wrote the CLI that consumes our minty-fresh public.json. The problem arose after my first test deployment. Excited, I navigated to my domain …

… and remembered painfully why we must specify our file types. AWS inferred my assets were all of type application/octet-stream because I failed to set the content type when I uploaded to S3. I could just specify each file’s type in flat JSON files like in the good ol’ days, but there must be a better way.

To get the content type of a file, I follow the Rust cookbook basically verbatim:

fn get_mime_type(filename: &str) -> Result<Mime, &'static str> {
    let parts: Vec<&str> = filename.split('.').collect();
    let res: Mime = match parts.last() {
        Some(v) => match *v {
            "css" => mime::TEXT_CSS,
            "html" => mime::TEXT_HTML,
            "js" => mime::APPLICATION_JAVASCRIPT,
            // ...
            &_ => mime::TEXT_PLAIN,
        },
        None => mime::TEXT_PLAIN,
    };
    Ok(res)
}

filename above is generated by prepending the absolute path to our root project directory to each entry in our public.json array. This absolute path can be passed to our CLI as a required argument. But the real hiccup comes once we’re ready to upload a file, as in this (abridged) example:

let mime: Mime = get_mime_type(&filename)
    .expect("Could not get media type");
let body: ByteStream =
    ByteStream::from_path(Path::new(&filename)).await?;
// `client` is of type aws_sdk_s3::Client
client.put_object()
    .content_type(mime.to_string())
    .bucket(bucket)
    .key(key)
    .body(body)
    .send()
    .await?;

key above comes directly from public.json. If we wanted to sneakily remove the file extension from uploaded assets, we would modify key before the call to put_object().

But how do we know which file extensions to remove? We could potentially remove all file extensions of a specific content type (e.g. text/html), but that’s probably casting too wide a net. We could hardcode specific file names (e.g. app.html), but then we may as well just hardcode the values in JSON again.

To me, the simpler option was to add an optional --subdirectory flag to the CLI that will prepend a string to each static asset specified in public.json.

∫ s3-deploy modified -p /Local/static/assets

2 files modified recently:
   build/bundle.js
   index.html
∫ s3-deploy yolo -b example.com -p /Local/static/assets -s app

Upload success for app/build/bundle.js
   Entity tag "<REDACTED>"
Upload success for app/index.html
   Entity tag "<REDACTED>"
∫

Now, you can navigate to example.com/app to see your beautiful static site!

To get started using s3-deploy, you can find it on Github. To see a static site deployed with s3-deploy, you’re lookin’ at one!

Block Quote End the Post

We, life’s pride and cared-for crown,
Have lost that cheer and charm of earth’s past prime:
Our make and making break, are breaking, down
To man’s last dust, drain fast towards man’s first slime.

— Gerard Manley Hopkins, “The Sea and the Skylark”