tl;dr Create our own testbed to measure the power consumption for running the test suite of common open source libraries. The testbed is CI/CD pipeline in which the Runners are on a Kubernetes cluster dedicated server (for now) that has access to the environmental data agent API or is monitored by a tool such as Scaphandre so that we can access or track the underlying power consumption. Each test run should report on power consumption so it can be further analyzed by the research team. The configuration of each test-run/test-subject will also be conducted by the research team.

• The researcher is configuring the various open-source libraries as test cases that can be run on the test infrastructure. The aim of the researcher is to review the different research subjects in regards to their power consumption. For the researcher, being able to get the data from each test run is the most important so it can be analyzed and further visualized.

• The infrastructure operator is the person responsible for configuring and running the CI/CD infrastructure, most likely within a Kubernetes cluster. This person wants to have it easy to expose the power consumption to the test environment for researchers and developers to use. It should be easy to set up, easy to monitor and to identify inaccuracies in the actually reported power consumption.

• The developer is the person building or maintaining an open-source library. For them, it should be very visible which increase or decrease in power consumption each new test execution is creating so that they can determine if their code is becoming more or less efficient. The resulting metrics should be reliable, consistent, and investigatable by the developer. Historic analysis from test-run to test-run should also be easy to do, so that the developer may understand the correlation between code changes and changes in power consumption.

• Test infrastructure: What is being defined and built within the scope of this project is a test infrastructure for open source libraries. This can be Jenkins or GitLab to give an example.

• Test run: The execution of a test suite within the test infrastructure

• Open-source library: As part of the project we have selected a few open source libraries that we would like to analyze in regards to their power consumption, examples can be Drupal or React

• Test suite: An integration or unit test suite, is a grouping of many tests in one suite, often with the aim to test the entire functionality of the library in one test run

• I want to be able to take an existing open-source library, with the unit or integration tests and easily configure a test run for it on the test infrastructure

• Optional: I want to be able to configure test runs without having to create a dedicated repository, without coding my own test setup

• I want to be able to trigger test runs manually to test the power consumption of a specific open-source library

• I want to be able to trigger a period or batch-based test run (e.g. running the same test for 24 hours 500 times)

• I want to be able to download the power consumption metrics & measurements for each test run in a computer-readable format (JSON, CSV, ...)

• I want to be able to see the power consumption metrics for each test within a test suite in Watt/hours

• I want to be able to see the total power consumption in Watt/hours for the entire test run

• I want to be able to install the power consumption measurement components (’plugins’) into my infrastructure with a few lines of commands
• Gitlab Plugin
• Kubernetes ‘Sidecar’
• Github Actions
• Jenkins Plugin

• I want to be able to still measure the power consumption even if the underlying or required APIs are not available in my infrastructure
• e.g. if the environmental data agent can not collect either data from the operating system (e.g. using RAPL) and there is no power consumption data exposed within the underlying virtualized infrastructure/data center, we will work with some constants/hardware specific power consumption data instead

• I want the additional measuring infrastructure to have a minimal additional resource footprint in my infrastructure

• I want to be able to see the power consumption of each test run of my code or changes to my code

• I want to be able to compare the power consumption of the latest test run with previous test runs

• I want to be able to see the power consumption per test in my test suite

• Optional: I want to receive an alert when there are significant changes in power consumption from one test run to another (e.g. 10% increase)