Using BiDirectional Protocol support in Selenium 4 to measure browser resource utilization

BiDi Protocol in Selenium 4 - not just network and console logs!

Previously, we showed how to record network requests and console logs in Selenium 4 via the currently-in-draft BiDirectional (or BiDi) protocol, but that was far from the only new capability that the new BiDi-related APIs provide! In this article we’ll show how to use Selenium 4 to inspect browser performance and resource-utilization metrics, using examples written in Scala.

Collecting Browser Performance and Resource Utilization data in Selenium 3

In previous versions of Selenium, there were no dedicated APIs for retrieving performance-related information from the browser - making it difficult to obtain performance data from the browser session while running your tests. You could open the DevTools while the browser was still open, but that approach breaks down if you want to automate metric collection and/or collect metrics periodically.

Performance Metrics and Memory Profiling in Selenium 4

To demonstrate the new APIs for browser performance introduced in Selenium 4, we’ll use Scala with Selenium’s Java library.

BiDi support in action

To start, we instantiate our WebDriver and create a buffer to hold performance metric information:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.devtools.v95.performance.model.Metric
import scala.collection.mutable.ListBuffer

val driver = new ChromeDriver()

val metricsByTimestamp = ListBuffer.empty[(Long, Seq[Metric])]

val devTools = driver.getDevTools

devTools.createSessionIfThereIsNotOne()

In the above block, we’re launching Chrome and opening a connection to it using the CDP (in a future release Selenium will use the BiDi protocol).

To pull and store performance metrics, we first need to enable metrics collection. Then we’ll create a collectMetrics method for pulling the metrics from the browser and storing them in our metricsByTimestamp buffer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import java.util.Optional
import org.openqa.selenium.devtools.v95.performance.Performance
import scala.collection.JavaConverters._

// See the note below for a description of the parameter for Performance.enable
devTools.send(Performance.enable(Optional.empty()))

def collectMetrics(): Unit = {
   val timestamp = System.currentTimeMillis()
   val metrics = devTools.send(Performance.getMetrics).asScala

   metricsByTimestamp.append(timestamp -> metrics)
}

devTools.send(Performance.getMetrics) sends a request to the browser for performance metrics, returning a list of Metric objects that in our case are added to the metricsByTimestamp buffer.

In addition to performance metrics, memory profiling can be enabled with just the following:

1
2
3
4
5

import java.util.Optional
import org.openqa.selenium.devtools.v95.memory.Memory

// See the note below for a description of the parameters of Memory.startSampling
devTools.send(Memory.startSampling(Optional.empty(), Optional.empty()))

Pulling the sampling information can be done with devTools.send(Memory.getAllTimeSamplingProfile), which we’ll use later on.

Putting together all the earlier code snippets, you can test against any website you’d like and print out the collected performance information with the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

import java.util.Optional
import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.devtools.v95.memory.Memory
import org.openqa.selenium.devtools.v95.performance.Performance
import org.openqa.selenium.devtools.v95.performance.model.Metric
import scala.collection.JavaConverters._
import scala.collection.mutable.ListBuffer

val driver = new ChromeDriver()

val metricsByTimestamp = ListBuffer.empty[(Long, Seq[Metric])]

val devTools = driver.getDevTools

devTools.createSessionIfThereIsNotOne()

// Begin memory sampling immediately
devTools.send(Memory.startSampling(Optional.empty(), Optional.empty()))

devTools.send(Performance.enable(Optional.empty()))

// Pull performance metrics and store them in the metricsByTimestamp buffer
def collectMetrics(): Unit = {
   val timestamp = System.currentTimeMillis()
   val metrics = devTools.send(Performance.getMetrics).asScala

   metricsByTimestamp.append(timestamp -> metrics)
}

// Pull metrics before navigating to have a point of reference to compare with
collectMetrics()

// Replace this string with the site you'd like to test
driver.get("https://reflect.run")

collectMetrics() // Pull the metrics again after navigating

// Retrieve the sampled memory information
val samplingProfile = devTools.send(Memory.getAllTimeSamplingProfile)

// Log the recorded performance metrics and memory information
metricsByTimestamp.foreach {
   case (timestamp, metrics) =>
      val metricStrings = metrics.map {
         metric => s"${metric.getName} -> ${metric.getValue}"
      }

      val message =
         s"""
            |$timestamp:
            |${metricStrings.mkString(System.lineSeparator())}
            |""".stripMargin

      println(message)
}

samplingProfile.getSamples.asScala.foreach { sample =>
   val message =
      s"""
         |Size of heap allocation: ${sample.getSize} bytes
         |Total bytes attributed to this sample: ${sample.getTotal} bytes
         |""".stripMargin

   println(message)
}

// Shut down the browser
driver.quit()

If you run the above code you should see the collected metrics printed out from before and after the page was loaded (list of metrics abridged for brevity):

1638229293144:
Documents -> 2
Nodes -> 8
JSHeapUsedSize -> 983432
JSHeapTotalSize -> 2043904
...

1638229295705:
Documents -> 37
Nodes -> 2824
JSHeapUsedSize -> 3024696
JSHeapTotalSize -> 4808704
...

Followed by the heap allocation information printed out afterwards (abridged for brevity):

Size of heap allocation: 128 bytes
Total bytes attributed to this sample: 131072 bytes

Size of heap allocation: 896 bytes
Total bytes attributed to this sample: 131072 bytes
...

Potential Applications

A more practical application for using this information than printing to console would be to compare the resource utilization of your web application over time. This could be done by saving the metrics to a storage solution such as AWS S3, or streaming the data into a solution such as AWS Kinesis for further processing/aggregation before storage.

Comparing the stored metrics with the latest run could be used to identify potential memory leaks or other performance regressions in your application.

Limitations

Like the other BiDi-related APIs in Selenium 4, the browser support for these new APIs is unfortunately limited to a subset of browsers - currently only Chrome and Microsoft Edge support them.

Additionally, while these APIs are supposed to use the BiDi protocol, they currently use the Chrome DevTools Protocol because the BiDi protocol is still in a draft state.

Lastly, while collecting performance information can be useful as part of testing with Selenium, the authors of the library themselves discourage using it for performance testing. That’s not to say the data isn’t worth collecting, but it is likely insufficient on its own to reliably identify broader performance issues (responsiveness, latency, etc.).

Conclusion

The new performance-related APIs in Selenium 4 can provide testers more information on their application in the browser than previous versions, but the currently limited browser support may hamper its utility for larger organizations.