HashiCorp 內 scale 的方法
source link: https://blog.gslin.org/archives/2023/08/22/11306/hashicorp-%e5%85%a7-scale-%e7%9a%84%e6%96%b9%e6%b3%95/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
HashiCorp 內 scale 的方法
去日本前在 Hacker News 上看到「Squeeze the hell out of the system you have」這篇，用作者的名字翻了一下 LinkedIn，看起來講的是 HashiCorp 的 SRE 事情：「Dan Slimmon」。
看的時候可以注意一下，文章裡面的觀點未必要認同，大多是他自己的看法或是想法，但裡面提到很多發生的事情，可以知道 HashiCorp 內目前搞了什麼東西。
從 LinkedIn 的資料可以看到他從 2019 就加入 HashiCorp 了，所以文章一開頭這邊講的同事應該就是 HashiCorp 的同事：
About a year ago, I raised a red flag with colleagues and managers about Postgres performance.
往下看可以看到他們有遇到 PostgreSQL 的效能問題，然後每次都是以 scale up (加大機器) 的方式解決，考慮到 HashiCorp 的產品線，我會猜應該是 Terraform Cloud 這個產品線遇到的狀況。
然後在後面提到的解法則是提到了 codebase 是 Rails，他們花了三個月的時候不斷的重複 profiling + optimizing，包括 SQL 與 PostgreSQL 的設定：
Two engineers (me and my colleague Ted – but mostly Ted) spent about 3 months working primarily on database performance issues. There was no silver bullet. We used our telemetry to identify heavy queries, dug into the (Rails) codebase to understand where they were coming from, and optimized or eliminated them. We also tuned a lot of Postgres settings.
另外一組人則是弄了 read-only replication server，把 loading 拆出去：
Two more engineers cut a path through the codebase to run certain expensive read-only queries on a replica DB. This effort bore fruit around the same time as (1), when we offloaded our single most frequent query (a SELECT triggered by polling web clients).
這兩個方法大幅降低了資料庫的 peak loading，從 90% 降到 30%：
These two efforts together reduced the maximum weekly CPU usage on the database from 90% to 30%.
可以看到都還沒用到 sharding 的技巧，目前硬體的暴力程度可以撐很久 (而且看起來是在沒有投入太多資源在 DB-related tuning 上面)，快撞到的時候也還可以先用 $$ 換效能，然後投入人力開始 profiling 找問題...
Related
HashiCorp 推出 Boundary 與 Waypoint
HashiCorp 推出了兩個新的產品。 第一個是 Boundary，這個產品看起來還很早期，但看起來想要實做 BeyondCorp 的一些想法 (可以參考之前寫的「Google 推的 BeyondCorp」)：「Announcing HashiCorp Boundary」，這在 Hacker News 上有討論串，而且 HashiCorp 的老大 Mitchell Hashimoto 也有出來回答一些問題：「HashiCorp Boundary (hashicorp.com)」。 這個產品讓他慢慢發展起來應該會是不錯的 open-source implementation，不過傳統的 castle 在小一點的場合也還算夠用... 另外一個是 Waypoint，看起來是個 CI/CD 類的產品：「Announcing HashiCorp Waypoint」，在 Hacker News 上也有討論串，而且同樣的 Mitchell Hashimoto 也有在裡面回覆一些東西：「Waypoint: Build, deploy, and release applications across any platform (hashicorp.com)」。 我自己看了一下設定的方式以及一些畫面，然後再回頭看 Hacker News…
October 20, 2020In "Computer"
Terraform 進入 1.0 版
HashiCorp 的東西都還蠻有品質的 (相較於其他的類似方案)，其中有一個觀察到的特性是，如果他門把版本維持在 0.x，代表他們自己對軟體的品質與成熟度都不算滿意。 而 Terraform 算是他們家對於基礎建設資源管理所提出的方案 (尤其是雲端時代的資源)，不過之前最頭痛的，每次升級就會發現有些基本的概念會被改掉，所以不是改改關鍵字就可以解決。 這次 Terraform 推出 1.0 版，代表了他們認為產品有一定的成熟度：「Announcing HashiCorp Terraform 1.0 General Availability」，另外在 GitHub 上面的「Release v1.0.0 · hashicorp/terraform」也可以看到： Terraform v1.0 is an unusual release in that its primary focus is on stability, and it represents the culmination of several years of work in previous major…
June 12, 2021In "Computer"
Leave a Reply
Your email address will not be published. Required fields are marked *
Comment *
Name *
Email *
Website
Notify me of follow-up comments by email.
Notify me of new posts by email.
To respond on your own website, enter the URL of your response which should contain a link to this post's permalink URL. Your response will then appear (possibly after moderation) on this page. Want to update or remove your response? Update or delete your post and re-enter your post's URL again. (Learn More)
Post navigation
Recommend
-
103
vaultingkube
-
96
Consul ESM (External Service Monitor) This project provides a daemon to run alongside Consul in order to run health checks for external nodes and update the status of those health checks in the catalog. It can also manage updating the coo...
-
83
Vault Please note: We take Vault's security and our users' trust very seriously. If you believe you have found a security issue in Vault, please responsibly disclose by contacting us at
-
118
raft is a Go library that manages a replicated log and can be used with an FSM to manage replicated state machines. It is a library for providing
-
56
HashiCorp、Contino企业Terraform推荐实践指南分享
-
88
除非特别声明，此文章内容采用知识共享署名 3.0许可，代码示例采用Apache 2.0许可。更多细节请查看我们的服务条款。
-
61
除非特别声明，此文章内容采用知识共享署名 3.0许可，代码示例采用Apache 2.0许可。更多细节请查看我们的服务条款。
-
70
README.md vault-init The vault-init service automates the process of initializing and
-
43
There are a few things I dislike about the programming industry. Much of what programmers do is driven by fads and trends. There is a lot of cargo culting with little critical analysis. This is especially true when it comes to...
-
58
README.md HCL
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK