Uber 對 Golang GC 的調整
source link: https://blog.gslin.org/archives/2022/01/13/10503/uber-%e5%b0%8d-golang-gc-%e7%9a%84%e8%aa%bf%e6%95%b4/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Uber 對 Golang GC 的調整
Hacker News 上看到「How We Saved 70K Cores Across 30 Mission-Critical Services (Large-Scale, Semi-Automated Go GC Tuning @Uber)」這篇，講 Uber 的人怎麼調整 Golang 的 GC，在 Hacker News 上的討論「Large-scale, semi-automated Go GC tuning (uber.com)」也有些東西再講。
一開始的方法是動態一直調整 GOGC 的值：
Our initial approach was to have a ticker to run every second to monitor the heap metrics, and then adjust GOGC value accordingly.
但這個方法的 overhead 太重：
The disadvantage of this approach is that the overhead starts to become considerable, because in order to read heap metrics Go needs to do a STW (ReadMemStats) and it is somewhat inaccurate, because we can have more than one garbage collection per second.
後來的方法是利用 SetFinalizer 來做 (然後這段 code 不知道為什麼是用圖片...)：
Luckily we were able to find a good alternative. Go has finalizers (SetFinalizer), which are functions that run when the object is going to be garbage collected. They are mainly useful for cleaning memory in C code or some other resources. We were able to employ a self-referencing finalizer that resets itself on every GC invocation. This allows us to reduce any CPU overhead.
不過 Hacker News 上有些人也很驚訝於 30 個 service 用掉 70K cores 這件事情，以 Uber 的服務來說算是比預想多不少數字，而且這只是跑 Golang，而且這次省下來的部份...
另外在 Hacker News 上也有人提到 Golang 有在思考 soft memory limit 的設計，也值得看一看：「runtime/debug: soft memory limit #48409」、「Proposal: Soft memory limit」。
Related
Go 1.9 的 GC 改善
Update：被提醒後仔細看了一下，是 1.8 預設生效 (但保留選項切回來 debug)，如果沒問題的話 1.9 把舊的方式拔乾淨： Assuming things go smoothly, we will remove stack re-scanning support when the tree opens for Go 1.9 development. 標題就不改了... 以下原文。 在「Sub-millisecond GC pauses」這邊看到的。Golang 想辦法將 GC 造成的影響降低：「Proposal: Eliminate STW stack re-scanning」。 目標是解決最大的 GC pause 來源： As of Go 1.7, the one remaining source of…
October 29, 2016In "Computer"
Java 17 (JDK 17)，新的 Java LTS 版本 (然後來看 GC)
Java 17 (JDK 17) 釋出，這是 Oracle 本家新的 LTS 版本，引用的是 jdk-dev 的 mailing list：「Java 17 / JDK 17: General Availability」。另外在 Hacker News 上的討論可以翻一下：「Java 17 / JDK 17: General Availability (java.net)」。 上一個 LTS 版本是 Java 11，所以很自然的也會有從 Java 11 之後的新功能說明：「JEPs in JDK 17 integrated since JDK 11」。 對於只是拿來用，而不是拿來開發的人來說，我的重點都放在 JVM 的 GC 效能以及特性。 從…
September 18, 2021In "Computer"
Instagram 解決 Cassandra 效能問題的方法
在解決 Cassandra 效能問題中大概就 ScyllaDB 特別有名，用 C++ 重寫一次使得效能大幅改善。而 Instagram 的人則是把底層的資料結構換掉，改用 RocksDB (這公司真的很愛自家的 RocksDB...)：「Open-sourcing a 10x reduction in Apache Cassandra tail latency」。 主要原因是他們發現 Cassandra 在處理資料的部份會有 JVM 的 GC 問題，而且是導致 Cassandra 效能差的主要原因： Apache Cassandra is a distributed database with it’s own LSM tree-based storage engine written in Java. We found that the components…
March 9, 2018In "Cassandra"
Leave a Reply Cancel reply
Your email address will not be published. Required fields are marked *
Comment
Name *
Email *
Website
Notify me of follow-up comments by email.
Notify me of new posts by email.
To respond on your own website, enter the URL of your response which should contain a link to this post's permalink URL. Your response will then appear (possibly after moderation) on this page. Want to update or remove your response? Update or delete your post and re-enter your post's URL again. (Learn More)
Post navigation
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK