ORES/Metrics
 Warning: | The ORES infrastructure is being deprecated by the Machine Learning team, please check wikitech:ORES for more info. |
We collect some metrics in the Python backend, and forward them to Grafana via a statsd endpoint. See wikitech:ORES/Metrics for the statsd endpoints used in Wikimedia installations of ORES.
precache_request - Incremented once for each "precache" score that is requested. Note that this is not the same scale as scores_request.
revision_scored - Unused.
scores_request - Incremented once for each non-"precache" score that is requested. (TODO: Give a more descriptive name, like "external_score_request")
score_processed - Incremented after each worker scores a revision (for each model that is scored). This is not called when a score is cached or found to be in-progress from a recent request.
score_processor_overloaded - Sent when a request is rejected due to the Celery queue exceeding the configured maximum queue size. We also return a "503 server overloaded" to the client.
score_cache_hit - Incremented for each revision score looked up in the Redis cache. (Note that the cache lookup is circumvented for requests that include "?features")
score_cache_miss - Incremented for each score cache lookup that fails. (Note that the cache lookup is circumvented for requests that include "?features")
score_errored - Incremented each time a score cannot be calculated because of a nonexistent revision ID, timeout, or other errors.
score_timed_out - Incremented when score processing takes longer than the scoring system's configured timeout (15 seconds as of Oct 2016).
datasources_extracted - Unused. TODO: ORES PR #176
precache_scores - Unused.
precache_score - Incremented within the precached utility when a score is successfully precached. (not used in ChangePropagation)
precache_scoring_error - Incremented within the precached utility when a score errors. (not used in ChangePropagation)
TODO:
- One more pass to tighten up how each event stream maps to a score e.g. per model.