Using S3 for Digital Goods Distribution

2015-10-15 (posted in blog)

I recently ran into a situation where a client of mine (my only “client,” who also happens to be my Mom) wanted to sell a digital video on her website. Until now, everything she sold had been a physical product, so I was able to get away with simple Paypal forms on her web store. Someone would click the “Buy Now” button, and Paypal would email my client so she knew to package and ship the product.

That workflow is fine for physical goods, but doesn’t really hold up for digital downloads—in theory my client could still “package and ship” the video by sending it as an email attachment, but the file is too large for that to be practical. A better option would be for me to put the video on the server, and let my client send the link out to people who had purchased it. The problem with that is that the audience for this particular video is a fairly tight-knit community, and it probably wouldn’t be long before the link to the video was shared (either intentionally or not) which would render the whole purchase step no longer necessary. Protecting the file with a password would be marginally better, but the password could be shared just as easily as the link.

My first thought was to develop some kind of home-grown expiring link system along these lines (pseudocode):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# Admin Tool for Client
func generate_temporary_link(){
  token = generate_random_token();
  database.insert({Token: token, Created: Date.now()});
  return "http://download.url/?token=" + token;
}

# Download Handler for Purchaser
func download_file(token){
  entry = database.select({Token: token});
  if(Date.now() - entry.Created > DateSpan.FromDays(3)){
    throw("Sorry, your link has expired.");
  }
  return FileStream.From("/private/local/path/to/video.mp4");
}

My only issue with that approach is that it would be a lot of friggin’ work. I’d have to define the database schema, create the admin service, do date math (I really hate date math), and basically write a bunch of code that seemed like it was solving a problem that should have already been solved before.

I was pretty sure that my client wouldn’t mind paying a nominal fee per download to utilize an existing service that provided expiring download links, so I did a little research to see what was already out there. These are some of the options I found (in no particular order) along with the reasons why they weren’t very appealing to me:

• Direct Digital Delivery - The one-time fee of $100 is, while quite steep, better than a monthly fee, but ideally I wanted a small per-transaction fee. This is also a self-hosted solution and feels like it might be a little beefy (definitely overkill for my use-case of a single product for a single client). Learning how to install and configure it would probably take more time than just writing the simple app I proposed above.

• DLGuard - Maybe it’s petty of me, but the website looks like it was generated by a template engine that was written in the 90’s. The fake software box art doesn’t inspire confidence. Also has a ton of features that would be overkill for what I wanted. I couldn’t find the pricing listed anywhere at first glance and didn’t bother digging too deep for it.

• PayLoadz - Monthly fee is a no-go (plus per-transaction fees on top of that). This is a low-traffic client that may not even have any sales in a given month.

• Sellify - The pricing scheme here is right, a 5% per-transaction fee. The problem is you have to use their storefront to sell your product—there would be no way to integrate the new product into the client’s existing web store (other than a link to this second storefront which would be confusing for the user).

• FlickRocket - This also has a per-transaction fee, but for download-to-own videos that fee is $2.50, which would be a pretty big chunk of the selling price. It also forces you to use their storefront for purchases.

• Pulley - Pulley was very, very close to what I wanted. They can integrate into an existing website, you get paid via Paypal, and secure links to the content are automatically emailed to the purchaser—the only problem is that they charge a monthly fee (with no transaction fees at all). If they had a per-transaction plan with reasonable rates I’d have given them a shot.

After being disappointed by most of the offerings in this space (I wonder if it’s worth spinning up a competitor of my own?), I decided to build my own home-grown solution after all.

My plan was to store the video in a non-public S3 bucket and then write some kind of expiring-link proxy to it. It was while I was looking for pre-existing S3 proxies that I stumbled upon this little nugget in the AWS.S3 getSignedUrl examples for the Javascript AWS SDK:

1
2
3
4

// Passing in a 1-minute expiry time for a pre-signed URL
var params = {Bucket: 'bucket', Key: 'key', Expires: 60};
var url = s3.getSignedUrl('getObject', params);
console.log('The URL is', url); // expires in 60 seconds

Holy crap! Amazon S3 already supports expiring links to content! This means all I need to do is give my client a tool to generate those links. No database schemas, no download proxying, and (best of all) no date math! Well, maybe a little bit of date math, but it’s the kind that you can do with a simple lookup.

Usually I’d use Perl for something like this, but I’ve been on a Node.js/CoffeeScript kick lately. In retrospect PHP probably would have been best but I don’t actually know any PHP so this is what I ended up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

fs = require 'fs'
express = require 'express'
auth = require 'basic-auth'
aws = require 'aws-sdk'
bitlygen = require 'bitly'

app = express()

fs.readFile "#{ __dirname }/config.json", 'utf8', (err, data) ->
  if err?
    console.log err
  else
    config = JSON.parse data
    s3 = new aws.S3 config.aws
    bitly = new bitlygen config.bitly.username, config.bitly.apikey

app.get '/get-url/:folder/:key', (req, res) ->
      creds = auth req
      if creds? and creds.name == config.authuser and creds.pass == config.authpass
        url = s3.getSignedUrl 'getObject',
          Bucket: config.bucket
          Key: req.params.folder + '/' + req.params.key
          Expires: 259200 # 3 days in seconds
        bitly.shorten url, (err, resp) ->
          if err?
            console.log err
            res.status(500).send 'Error 500 - Contact Rudi'
          else
            res.send resp.data.url
      else
        res.setHeader 'WWW-Authenticate', "Basic realm=\"#{ req.hostname }\""
        res.status(401).send 'Error 401 - Unauthenticated'

server = app.listen 8123, ->
      console.log 'Listening on port 8123'

The script requires a config.json file next to it that looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{
  "authuser": "myuser",
  "authpassword": "mypassword",
  "bucket": "my-private-bucket-name",
  "bitly":
  {
    "username": "bitlyusername",
    "apikey": "MY_BITLY_API_KEY"
  },
  "aws":
  {
    "accessKeyId": "MY_AWS_ACCESS_KEY_ID",
    "secretAccessKey": "MY_AWS_SECRET_ACCESS_KEY",
    "region": "us-east-1"
  }
}

The script is written in a way that it can generate expiring links to any file in the configured bucket. My client visits the site at http://admin.site/clientname/video.mp4 and the script requests the signed link from Amazon (using the key clientname/video.mp4), telling it to expire the link in 3 days. The link is shortened using Bitly (the AWS link is quite long and would probably be broken onto multiple lines by some email programs which could cause problems), and displays the Bitly link to the client. The client can then copy that link and send it to someone who purchased their video. Each purchaser gets their own link which will stop working after 3 days.

There are still some downsides to this method—it would be nice if the link could be emailed to the client automatically, for example, and the error message displayed by Amazon after the link expires is pretty unfriendly, but for low-volume sales I’m pretty satisfied with this as a low-effort completely free solution to the problem.

At least until I spin up my own simple digital sales platform to compete with the disappointing and/or bloated options already out there. Or not.

Short Permalink for Attribution: rdsm.ca/uwx3u

Comments