Create a Redis Leaderboard with Golang
source link: https://www.vultr.com/docs/create-a-redis-leaderboard-with-golang
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Introduction
Redis sorted set (ZSET
) is a powerful data structure that allows you to create highly responsive and scalable leaderboards. Traditionally, the ZSETs
were primarily associated with gaming applications. However, you can use the Redis leaderboards for many applications in today's evolving IT industry.
For instance, you can create a scoreboard to log the total revenue generated by each salesperson and their rank compared to other staff. This enhances healthy competition in your sales department. Similarly, you can implement a leaderboard in a fitness tracking app to encourage members to complete their goals. For example, you can log their step counts or any other exercise they want to complete in a certain time period.
You can store and calculate ranks with relational database management systems like MySQL on a small scale. However, disk-based databases perform poorly and are prone to scalability issues when million of users' records are involved. The Redis in-memory database server performs optimally for these kinds of operations, and its sorted set data structure can handle a large load efficiently.
In this guide, you'll use the Redis ZSET
data type to create a leaderboard with Golang on your Linux server.
Prerequisites
To proceed with this tutorial, ensure you've the following:
1. Create a main.go
File
In this tutorial, you'll implement a web application that listens for incoming POST
and GET
requests to add and retrieve information in a Redis ZSET
.
The application will log step count data for different users participating in a fitness tracking competition. In the end, you'll be able to send a curl
POST command to add steps for a user and a GET
command to list participating members and their rankings.
Begin by creating a
project
folder for your application. This separates your Golang source code files from the rest of the Linux files to make troubleshooting easier if you encounter errors in the future.$ mkdir project
Navigate to the new
project
directory.$ cd project
Next, open a new
main.go
file. This file runs themain()
function, which fires when you start the application.$ nano main.go
Then, enter the information below into the
main.go
file.package main import ( "encoding/json" "fmt" "net/http" "github.com/go-redis/redis" ) func main() { http.HandleFunc("/scores", httpHandler) http.ListenAndServe(":8080", nil) } func httpHandler(w http.ResponseWriter, req *http.Request) { redisClient := redis.NewClient(&redis.Options{ Addr: "localhost:6379", Password: "", DB: 0, }) params := map[string]interface{}{} resp := map[string]interface{}{} var err error if req.Method == "GET" { for k, v := range req.URL.Query() { params[k] = v[0] } resp, err = getScores(redisClient, params) } else if req.Method == "POST" { err = json.NewDecoder(req.Body).Decode(¶ms) resp, err = addScore(redisClient, params) } enc := json.NewEncoder(w) enc.SetIndent("", " ") if err != nil { resp = map[string]interface{}{ "error": err.Error(), } } else { if encodingErr := enc.Encode(resp); encodingErr != nil { fmt.Println("{ error: " + encodingErr.Error() + "}") } } }
Save and close the
main.go
file when you're through with editing.In the above file you're importing the
encoding/json
package to format response data in JSON format. Then, you're usingfmt
to output any basic response to the user. The packagenet/http
provides HTTP methods to your application while the librarygithub.com/go-redis/redis
allows you to communicate to the Redis server.Under the
main()
function, you're using the statementhttp.HandleFunc("/scores", httpHandler)
to route incoming requests to thehttpHandler
function. Then, you're starting a web server on port8080
using the statementhttp.ListenAndServe(":8080", nil)
In the
httpHandler
function, you're connecting to the Redis server using the statementredisClient := redis.NewClient()...
. Next, you're creating a Golangmap
of[string]interface{}
by retrievingGET
andPOST
variables from thereq.URL.Query()
andreq.Body
functions.Then, you're using the Golang
if {...} else {...}
statement to forward incoming requests to either anaddScore(redisClient, params)
function or agetScores(redisClient, params)
function. In the next, step you'll create theaddScore
andgetScores
functions in separate files.
2. Create an add_score.go
File
In the previous main.go
function, you're redirecting any POST
request to an addScore()
function. In this step, you'll create the function insider an add_score.go
file.
Use
nano
to create theadd_score.go
file.$ nano add_score.go
Next, enter the information below into the
add_score.go
file.package main import ( "context" "github.com/go-redis/redis" ) func addScore(c *redis.Client, p map[string]interface{}) (map[string]interface{}, error) { ctx := context.TODO() nickname := p["nickname"].(string) steps := p["steps"].(float64) //Validate data here in a production environment err := c.ZAdd(ctx, "app_users", &redis.Z{ Score: steps, Member: nickname, }).Err() if err != nil { return nil, err } rank := c.ZRank(ctx, "app_users", p["nickname"].(string)) if err != nil { return nil, err } response := map[string]interface{}{ "data": map[string]interface{}{ "nickname": p["nickname"].(string), "rank": rank.Val(), }, } return response, nil }
Save and close the file.
In the above file, you're connecting to the Redis server to add a sorted set entry using the statement
c.ZAdd(ctx, "app_users", &redis.Z{ Score: steps, Member: nickname, }).Err()
. Theapp_users
is the name of your sorted set as stored in the Redis in-memory database. Then, you're capturing thesteps
from sample fitness users as a value for theScore
variable. You're then distinguishing the different users by populating theMember
variable with the different members'nicknames
.Towards the end of the file, you're using the
rank := c. ZRank(ctx, "app_users", p["nickname"].(string))
statement to get the rank of the member and then, you're return a map of[string]interface{}
to the calling function.Your
add_score.go
file primarily handles new entries to the Redis sorted set. To retrieve the entries, you'll create a new file in the next step.
3. Create a get_scores.go
File
When working with sorted sets, you can use different Redis functions to compute the members' ranks. For instance, you can return a list of all members with their scores arranged in descending order using the ZRevRangeWithScores()
function. Also, you can count total entries in a sorted set using the function ZCount()
.
In this step, you'll create a file that uses the two functions to retrieve and return members' scores from the Redis server.
Create the
get_scores.go
File.$ nano get_scores.go
Enter the information below into the
get_scores.go
file.package main import ( "context" "fmt" "strconv" "github.com/go-redis/redis" ) func getScores(c *redis.Client, p map[string]interface{}) (map[string]interface{}, error) { ctx := context.TODO() start, err := strconv.ParseInt(fmt.Sprint(p["start"]), 10, 64) if err != nil { return nil, err } stop, err := strconv.ParseInt(fmt.Sprint(p["stop"]), 10, 64) if err != nil { return nil, err } total, err := c.ZCount(ctx, "app_users", "-inf", "+inf").Result() //int64 if err != nil { return nil, err } scores, err := c.ZRevRangeWithScores(ctx, "app_users", start, stop).Result() //highest to lowest score if err != nil { return nil, err } data := []map[string]interface{}{} for _, z := range scores { record := map[string]interface{}{} rank := c.ZRank(ctx, "app_users", z.Member.(string)) if err != nil { return nil, err } record["nickname"] = z.Member.(string) record["score"] = z.Score record["rank"] = rank.Val() data = append(data, record) } countPerRequest := stop - start + 1 if stop == -1 { countPerRequest = total } response := map[string]interface{}{ "data": data, "meta": map[string]interface{}{ "start": start, "stop": stop, "per_request": countPerRequest, "total": total, }, } return response, nil }
Save and close the
get_scores.go
fileIn the above file you're parsing the
start
andstop
indices from the URL variables as submitted in aGET
request to determine the number of records to return from the Redis set.Next, you're using the
scores, err := c.ZRevRangeWithScores(ctx, "app_users", start, stop).Result()
statement to get the members and their respective ranks from the highest to lowest score.Then, you're looping through the
scores
array to append members' details to a datamap
that you're returning to the calling function. In each loop cycle, you're using therank := c.ZRank(ctx, "app_users", z.Member.(string))
statement to retrieve the rank of each member. In a Redis sorted set, the member with the leastscore
gets thelowest'rank
.Your Redis leaderboard application is now ready for testing.
4. Test the Redis Leaderboard Application
In this step, you'll add and retrieve members' scores from your Redis server by running curl
statements against your application's endpoint.
Before you do this, download the Redis package that you're using in your application.
$ go get github.com/go-redis/redis
Next, run the application. The following command starts a web server. Your application should now listen for incoming requests on port
8080
. Don't enter any other command on your currentSSH
terminal.$ go run ./
Connect to your server in a new terminal window and execute the following
curl
commands one by one to add entries into the Redis database.$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "steven", "steps": 2125}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "john", "steps": 300}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "jane", "steps": 1426}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "francis", "steps": 765}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "doe", "steps": 923}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "mary", "steps": 654}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "mark", "steps": 958}' $ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "peter", "steps": 1456}'
You should get the following response after executing each
POST
command above. The JSON response outputs the member'snickname
and therank
.... { "data": { "nickname": "jane", "rank": 1 } } ...
Next, run the following
GET
command to retrieve all records from your application. Set thestart
andstop
indices to0
and-1
respectively to return all members.$ curl -X GET "localhost:8080/scores?start=0&stop=-1"
You should now see the JSON response detailing all members, their scores, and ranks. The meta-information displayed at the end shows your set indices(
start
andstop
), total members found in the range of indices(per_request
), and all members in theapp_users
sorted set(total
).{ "data": [ { "nickname": "steven", "rank": 7, "score": 2125 }, { "nickname": "peter", "rank": 6, "score": 1456 }, { "nickname": "jane", "rank": 5, "score": 1426 }, { "nickname": "mark", "rank": 4, "score": 958 }, { "nickname": "doe", "rank": 3, "score": 923 }, { "nickname": "francis", "rank": 2, "score": 765 }, { "nickname": "mary", "rank": 1, "score": 654 }, { "nickname": "john", "rank": 0, "score": 300 } ], "meta": { "per_request": 8, "start": 0, "stop": -1, "total": 8 } }
Next, change the
start
andstop
indices to return only a sub-set from your sorted set. For instance, to retrieve only the first 3 items, run the command below with astart
index of0
and astop
index of2
$ curl -X GET "localhost:8080/scores?start=0&stop=2"
You should now get 3 records.
{ "data": [ { "nickname": "steven", "rank": 7, "score": 2125 }, { "nickname": "peter", "rank": 6, "score": 1456 }, { "nickname": "jane", "rank": 5, "score": 1426 } ], "meta": { "per_request": 3, "start": 0, "stop": 2, "total": 8 } }
Your application is working as expected. In this tutorial, you enter data manually using the
curl
command. In a production environment, you should supply data to your application from external data sources such as mobile apps connected to the fitness tracking wrist bands.
Conclusion
In this tutorial, you've created a Redis leaderboard application that returns data in JSON format with Golang on your Linux server. You've used the Redis ZSET
functions to add and retrieve the scores of members participating in a fitness tracking application.
Follow the links below to read more Golang tutorials:
- Secure a Golang Web Server with a Self-signed or Let's Encrypt SSL Certificate
- How to Create a Central Input Data Validator in Golang
- Designing a Modern API with Golang and MySQL 8 on Linux
Want to contribute?
You could earn up to $600 by adding new articles
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK