| | |
- __builtin__.object
-
- BenchmarkMetadata
- exceptions.Exception(exceptions.BaseException)
-
- InvalidOptionsError
- telemetry.internal.util.command_line.Command(telemetry.internal.util.command_line.ArgumentHandlerMixIn)
-
- Benchmark
class Benchmark(telemetry.internal.util.command_line.Command) |
| |
Base class for a Telemetry benchmark.
A benchmark packages a measurement and a PageSet together.
Benchmarks default to using TBM unless you override the value of
Benchmark.test, or override the CreatePageTest method.
New benchmarks should override CreateStorySet.
|
| |
- Method resolution order:
- Benchmark
- telemetry.internal.util.command_line.Command
- telemetry.internal.util.command_line.ArgumentHandlerMixIn
- __builtin__.object
Methods defined here:
- CreatePageTest(self, options)
- Return the PageTest for this Benchmark.
Override this method for PageTest tests.
Override, override CreateTimelineBasedMeasurementOptions to configure
TimelineBasedMeasurement tests. Do not override both methods.
Args:
options: a browser_options.BrowserFinderOptions instance
Returns:
|test()| if |test| is a PageTest class.
Otherwise, a TimelineBasedMeasurement instance.
- CreateStorySet(self, options)
- Creates the instance of StorySet used to run the benchmark.
Can be overridden by subclasses.
- CreateTimelineBasedMeasurementOptions(self)
- Return the TimelineBasedMeasurementOptions for this Benchmark.
Override this method to configure a TimelineBasedMeasurement benchmark.
Otherwise, override CreatePageTest for PageTest tests. Do not override
both methods.
- CustomizeBrowserOptions(self, options)
- Add browser options that are required by this benchmark.
- GetMetadata(self)
- GetTraceRerunCommands(self)
- Run(self, finder_options)
- Do not override this method.
- SetupBenchmarkDebugTraceRerunOptions(self, tbm_options)
- Setup tracing categories associated with debug trace option.
- SetupBenchmarkDefaultTraceRerunOptions(self, tbm_options)
- Setup tracing categories associated with default trace option.
- SetupTraceRerunOptions(self, browser_options, tbm_options)
- __init__(self, max_failures=None)
- Creates a new Benchmark.
Args:
max_failures: The number of story run's failures before bailing
from executing subsequent page runs. If None, we never bail.
Class methods defined here:
- AddCommandLineArgs(cls, parser) from __builtin__.type
- HasTraceRerunDebugOption(cls) from __builtin__.type
- Name(cls) from __builtin__.type
- ProcessCommandLineArgs(cls, parser, args) from __builtin__.type
- SetArgumentDefaults(cls, parser) from __builtin__.type
- ShouldDisable(cls, possible_browser) from __builtin__.type
- Override this method to disable a benchmark under specific conditions.
Supports logic too complex for simple Enabled and Disabled decorators.
Decorators are still respected in cases where this function returns False.
- ValueCanBeAddedPredicate(cls, value, is_first_result) from __builtin__.type
- Returns whether |value| can be added to the test results.
Override this method to customize the logic of adding values to test
results.
Args:
value: a value.Value instance (except failure.FailureValue,
skip.SkipValue or trace.TraceValue which will always be added).
is_first_result: True if |value| is the first result for its
corresponding story.
Returns:
True if |value| should be added to the test results.
Otherwise, it returns False.
Data descriptors defined here:
- max_failures
Data and other attributes defined here:
- options = {}
- test = <class 'telemetry.web_perf.timeline_based_measurement.TimelineBasedMeasurement'>
- Collects multiple metrics based on their interaction records.
A timeline based measurement shifts the burden of what metrics to collect onto
the story under test. Instead of the measurement
having a fixed set of values it collects, the story being tested
issues (via javascript) an Interaction record into the user timing API that
describing what is happening at that time, as well as a standardized set
of flags describing the semantics of the work being done. The
TimelineBasedMeasurement object collects a trace that includes both these
interaction records, and a user-chosen amount of performance data using
Telemetry's various timeline-producing APIs, tracing especially.
It then passes the recorded timeline to different TimelineBasedMetrics based
on those flags. As an example, this allows a single story run to produce
load timing data, smoothness data, critical jank information and overall cpu
usage information.
For information on how to mark up a page to work with
TimelineBasedMeasurement, refer to the
perf.metrics.timeline_interaction_record module.
Args:
options: an instance of timeline_based_measurement.Options.
results_wrapper: A class that has the __init__ method takes in
the page_test_results object and the interaction record label. This
class follows the ResultsWrapperInterface. Note: this class is not
supported long term and to be removed when crbug.com/453109 is resolved.
Class methods inherited from telemetry.internal.util.command_line.Command:
- Description(cls) from __builtin__.type
- main(cls, args=None) from __builtin__.type
- Main method to run this command as a standalone script.
Data descriptors inherited from telemetry.internal.util.command_line.ArgumentHandlerMixIn:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
|
class BenchmarkMetadata(__builtin__.object) |
| | |
Methods defined here:
- AsDict(self)
- __init__(self, name, description='', rerun_options=None)
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
- description
- name
- rerun_options
|
|