Nothing Special   »   [go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Prometheus integration to use remote writing #637

Open
tsenart opened this issue Jul 24, 2023 · 4 comments
Open

Upgrade Prometheus integration to use remote writing #637

tsenart opened this issue Jul 24, 2023 · 4 comments

Comments

@tsenart
Copy link
Owner
tsenart commented Jul 24, 2023

Limitations of the current exporter based approach

  1. Prometheus scrapes metrics from a running vegeta attack process and assigns timestamps to samples on its server. This means result timestamps aren't accurate (i.e. they're scraping time, not result time).
  2. Configuring Prometheus to scrape vegeta needs to happen out-of-band. That's a hassle!
  3. Since there's no coordination between a vegeta attack process and a Prometheus server, an attack process will finish before Prometheus has the chance to scrape the latest observations.

Instead, we should do something like this: https://k6.io/docs/results-output/real-time/prometheus-remote-write/

@flaviostutz
Copy link
Contributor

I think this is a good addition for distributed "attacks"!

While we don't implement this, for distributed load tests an alternative would be:

We still will have some of the issues mentioned above, but it can help in the meanwhile.

@flaviostutz
Copy link
Contributor

Would we implement this as part of the "attack" (changing the current implementation), or as a separate command?

In both situations we have to be careful about the latency introduced by the push part, so probably we have to make it run asynchronously, in a separate goroutine from the main processes.

What do you think?

@tsenart
Copy link
Owner Author
tsenart commented Jul 25, 2023

Sure, we'll have to be careful always when introducing changes to attack, this included! I'm leaning to change the existing implementation + introduce a separate prom-write sub-command.

@peterbourgon
Copy link
Contributor

Just want to note that distributed tests shouldn't rely on an etcd server, or any other kind of external coordinator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants