1623304 - Ping upload using the Firefox network stack
source link: https://bugzilla.mozilla.org/show_bug.cgi?id=1623304
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Ping upload using the Firefox network stack
Categories
(Toolkit :: Telemetry, task, P3)
Tracking
(bug RESOLVED as FIXED for Firefox 79)
mozilla79
People
(Reporter: janerik, Assigned: chutten|PTO)
References
Details
(Whiteboard: [telemetry:fog:m3])
FOG needs to upload pings after submission.
The Upload should use the Firefox network stack to do this.
This requires the new upload manager to land.
(the "depends on" bug is closed, but the changes are not in master yet)
I just had a chat with :baku for other reasons, he suggested looking at fetch
and fetchObserver
(usable in C++ as well, example) instead of using XHR for new code.
C++ via getting a JS Context, I notice : |
I wonder if there's a friendly and direct networking API in native that we can more easily use without nsI* or JSContext gymnastics in our Rust layer...
Getting a little more detailed with the requirements, this shall:
- Send pings when it is pleasant to do so. Scheduling is opportunistic and asynchronous and perhaps on idle. Especially since networking things might have to be main-thread.
- Control some headers
- We do not want anything hitting this endpoint that doesn't need to. This especially means Cookie headers, but extends to anything outside of Content-Type (json), Content-Encoding (gzip, please), Date, User-Agent (useful if something goes wrong), and the basic protocol-level stuff (looking at you, Host)
- We do want to add some extra things for our own purposes. They all start with
X-
so I don't expect that to be troubling.
While waiting for the dependencies to land I explored the problem space a bit and learned some things:
Viaduct
- The
viaduct
crate should probably re-exporturl::Url
so that we don't have to match theirurl
version. I filed an Issue - It's not strictly clear who should be initializing the backend or how. I went the route of adding some JS (
let viaduct = Cc["@mozilla.org/toolkit/viaduct;1"].createInstance(Ci.mozIViaduct); viaduct.Initialize();
) right before we call in to initnsITelemetry
(which is where we're currently init-ing FOG). I'm using the version of the viaduct impl that is currently up for review in bug 1628068 so I'm way too early to be asking questions like these... but it might still reveal that there's a timing thing we should pay attention to.
Glean SDK
- Getting the correct version of
glean_core
was a little tricky because I needed to specify it on both thefog
andglean
crates (int/c/g/
andt/c/g/api/
respectively). I wonder if there's anything we can do about that. And in the future when we'll be depending on theglean_sdk
crate for the Rust Specific Metrics API, what's to keep us from getting out of sync and accidentally vendoring 2 versions ofglean_core
? - The ping upload changes aren't usable from Rust at the moment because
PingRequest
isn't exported. See bug 1634745 - Where exactly should we be calling
glean.on_ready_to_submit_pings()
? Filed bug 1634754 for the documentation clarification, but in concrete terms I guess we should be calling it... inapi::initialize
?
Scheduling
viaduct
asserts if we try to do (sync) networking on the main thread (good!).- There's nothing stopping us from using
std::thread
(which is what I did in my exploration), but as :lina notes it only works for us until we want to dispatch runnables in which case we should use the quite tolerable-lookingnsIThread
stuff thatrkv
uses. And I can't help but notice aRefPtr<nsIThread>
could sit nicely insideAppState
.- Dumping to a bg thread like this is enough for now but I wonder if we'll ever in the future need to look into scheduling on idle. For now let's try our best not to schedule anything on a timer and instead allow ourselves to be triggered by activity elsewhere in the process (by say
Ping::submit
and startup? And IPC? Whatever we choose we must remember to document.)
- Dumping to a bg thread like this is enough for now but I wonder if we'll ever in the future need to look into scheduling on idle. For now let's try our best not to schedule anything on a timer and instead allow ourselves to be triggered by activity elsewhere in the process (by say
Anyhoo, that's as far as I can take this for now. Time to de-prioritize and unassign until we see some action in the dependencies.
Depends on getting glean_sdk to 30.1.0 so we can have a Vec<u8>
out of the ping request.
Current testing method is the following patch:
diff --git a/toolkit/components/glean/src/lib.rs b/toolkit/components/glean/src/lib.rs
index 4c8119a20dc0..53e30bfa722a 100644
--- a/toolkit/components/glean/src/lib.rs
+++ b/toolkit/components/glean/src/lib.rs
@@ -102,5 +102,12 @@ pub unsafe extern "C" fn fog_init(
}
}
+ // DO NOT MERGE
+ log::error!("Testing ping sending");
+ let ping = glean::metrics::Ping::new("ping_name", false, true, vec![]);
+ let success = ping.submit(None);
+ log::error!("Ping success? {}", success);
+ // DO NOT MERGE
+
NS_OK
}
diff --git a/toolkit/components/glean/src/api.rs b/toolkit/components/glean/src/api.rs
index a700df32a520..8d45ba404442 100644
--- a/toolkit/components/glean/src/api.rs
+++ b/toolkit/components/glean/src/api.rs
@@ -147,6 +147,7 @@ fn register_uploader() {
const SERVER: &str = "https://incoming.telemetry.mozilla.org";
let url = Url::parse(SERVER)?.join(&ping_request.path)?;
let mut req = Request::post(url);
+ req = req.header("X-Debug-ID", "chutten_fog_test")?;
for (&header_key, header_value) in ping_request.headers.iter() {
req = req.header(header_key, header_value)?;
}
Apply the patch, build and run Firefox, then check the Debug Ping Viewer.
Currently the pings come up empty (didn't add the PingRequest's body to the viaduct Request.), but if they come up at all then it's working.
We will eventually have a solution for what FOG does for Gecko when not in
Firefox Desktop, but for now skip the Android question entirely.
Depends on D76750
In the not-too-distant future we'll need to support configurations where we still build FOG into Gecko and not initialize it or its networking underpinnings. But for now we'll just not build anything at all on Android.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK