SmartWatts Formula

SmartWatts is a software-defined power meter based on the PowerAPI toolkit. SmartWatts is a configurable software that can estimate the power consumption of software in real-time. SmartWatts need to receive several metrics provided by HWPC Sensor :

The Running Average Power Limit (RAPL)
TSC
APERF
MPERF
CPU_CLK_THREAD_UNHALTED:REF_P (Sandy Bridge through Broadwell) or CPU_CLK_UNHALTED:REF_P (Skylake and newer)
CPU_CLK_THREAD_UNHALTED:THREAD_P (Sandy Bridge through Broadwell) or CPU_CLK_UNHALTED:THREAD_P (Skylake and newer)
LLC_MISSES
INSTRUCTIONS_RETIRED

These metrics are then used as inputs for a power model that estimates the power consumption of each software. The model can derive from the reality, each time the cpu-error-threshold is reached it learns a new power model, using the previous reports.

The choice of those specific metrics is motivated in SmartWatts: Self-Calibrating Software-Defined Power Meter for Containers

Installation

You can use the following script to install SmartWatts and HWPC Sensor.

DockerPypi

docker pull powerapi/smartwatts-formula

pip install smartwatts

Usage

For running the SmartWatts Formula you need: a Source and a Destination, a Sensor that provides HWPCReports and a configuration.

Source and Destination

For running SmartWatts we are using MongoDB as Source and InfluxDB as Destination as dockers containers.

To start a MongoDB instance via the command line

docker run -d --name mongo_source -p 27017:27017 mongo

And a InfluxDB instance

docker run -d --name influx_dest -p 8086:8086 influxdb:1.8

Sensor

HWPC Sensor is used in order to get HWPCReports. Start by installing the HWPC Sensor (see here) and start it (see here).

Parameters

Besides the basic parameters, the following ones are specific to SmartWatts:

Parameter	Type	CLI shortcut	Default Value	Description
`disable-cpu-formula`	`bool` (flag)	-	`false`	Disable CPU Formula
`disable-dram-formula`	`bool` (flag)	-	`false`	Disable RAM Formula
`cpu-rapl-ref-event`	`string`	-	`"RAPL_ENERGY_PKG"`	RAPL event used as reference for the CPU power models
`dram-rapl-ref-event`	`string`	-	`"RAPL_ENERGY_DRAM"`	RAPL event used as reference for the DRAM power models
`cpu-tdp`	`int`	-	`125`	CPU TDP (in Watt)
`cpu-base-clock`	`int`	-	`100`	CPU base clock (in MHz)
`cpu-base-freq`	`int`	-	`2100`	CPU base frequency (in MHz)
`cpu-error-threshold`	`float`	-	`2.0`	Error threshold for the CPU power models (in Watts)
`dram-error-threshold`	`float`	-	`2.0`	Error threshold for the DRAM power models (in Watts)
`learn-min-samples-required`	`int`	-	`10`	Minimum amount of samples required before trying to learn a power model
`learn-history-window-size`	`int`	-	`60`	Size of the history window used to keep samples to learn from
`sensor-reports-frequency`	`int`	-	`1000`	The frequency with which measurements are made (in milliseconds)

Running the Formula via CLI parameters

In order to run the Formula, you can execute one of the following command lines, depending on the installation you use:

DockerPip

docker run -t \
--net=host \
powerapi/smartwatts-formula --verbose \
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
--cpu-base-freq 1900 \
--cpu-error-threshold 2.0 \
--disable-dram-formula \
--sensor-reports-frequency 1000

python -m smartwatts \
--verbose \
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
--cpu-base-freq 1900 \
--cpu-error-threshold 2.0 \
--disable-dram-formula \
--sensor-reports-frequency 1000

In this configuration we are using MongoDB as source and InfluxDB as Destination. Some parameters values depend of your hardware. In particular, cpu-base-freq. You can obtain this value from CPU MHz field by using lscpu command.

Estimations' Storage

Your PowerReports will be stored on InfluxDB. You can watch them in a grafana by using the following tutorial.

Using shortcuts for parameters' names

You use - instead of --.

Running the Formula with Environment Variables

Parameters are defined by using the prefixes POWERAPI_, POWERAPI_INPUT_ and POWERAPI_OUTPUT_ in the names of Environment Variables. The following conventions are used:

POWERAPI_<PARAMETER_NAME>
POWERAPI_INPUT_<COMPONENT_NAME>_<PARAMETER_NAME>
POWERAPI_OUTPUT_<COMPONENT_NAME>_<PARAMETER_NAME>

where PARAMETER_NAME refers to names of parameters in upper case (e.g., VERBOSE, CPU_BASE_FREQ, COLLECTION) and COMPONENT_NAME to the name given to the different Sources and Destinations in upper case (e.g., PULLER and PUSHER_POWER).

Below you find an example for running the Formula with Docker and Pip:

DockerPip

docker run -t \
--net=host \
-e POWERAPI_VERBOSE=true \
-e POWERAPI_STREAM=true \
-e POWERAPI_CPU_BASE_FREQ=1900 \
-e POWERAPI_CPU_ERROR_THRESHOLD=2.0 \
-e POWERAPI_DISABLE_DRAM_FORMULA=true \
-e POWERAPI_SENSOR_REPORTS_FREQUENCY=1000 \
-e POWERAPI_INPUT_PULLER_MODEL=HWPCReport \
-e POWERAPI_INPUT_PULLER_TYPE=mongodb \
-e POWERAPI_INPUT_PULLER_URI=mongodb://127.0.0.1 \
-e POWERAPI_INPUT_PULLER_DB=test \
-e POWERAPI_INPUT_PULLER_COLLECTION=prep \
-e POWERAPI_OUTPUT_PUSHER_POWER_MODEL=PowerReport \
-e POWERAPI_OUTPUT_PUSHER_POWER_TYPE=influxdb \
-e POWERAPI_OUTPUT_PUSHER_POWER_URI=127.0.0.1 \
-e POWERAPI_OUTPUT_PUSHER_POWER_PORT=8086 \
-e POWERAPI_OUTPUT_PUSHER_POWER_DB=power_consumption2 \
powerapi/smartwatts-formula

export POWERAPI_VERBOSE=true
export POWERAPI_STREAM=false
export POWERAPI_CPU_BASE_FREQ=1900
export POWERAPI_CPU_ERROR_THRESHOLD=2.0
export POWERAPI_DISABLE_DRAM_FORMULA=true
export POWERAPI_SENSOR_REPORTS_FREQUENCY=1000
export POWERAPI_INPUT_PULLER_MODEL=HWPCReport
export POWERAPI_INPUT_PULLER_TYPE=mongodb
export POWERAPI_INPUT_PULLER_URI=mongodb://127.0.0.1
export POWERAPI_INPUT_PULLER_DB=test
export POWERAPI_INPUT_PULLER_COLLECTION=prep
export POWERAPI_OUTPUT_PUSHER_POWER_MODEL=PowerReport
export POWERAPI_OUTPUT_PUSHER_POWER_TYPE=influxdb
export POWERAPI_OUTPUT_PUSHER_POWER_URI=127.0.0.1
export POWERAPI_OUTPUT_PUSHER_POWER_PORT=8086
export POWERAPI_OUTPUT_PUSHER_POWER_DB=power_consumption2
python -m smartwatts

Running the Formula with a Configuration File

Below an example is provided by using MongoDB as Source and InfluxDB as Destination.

{
  "verbose": true,
  "stream": true,
  "input": {
    "puller": {
      "model": "HWPCReport",
      "type": "mongodb",
      "uri": "mongodb://127.0.0.1",
      "db": "test",
      "collection": "prep"
    }
  },
  "output": {
    "pusher_power": {
      "type": "influxdb",
      "uri": "127.0.0.1",
      "port": 8086,
      "db": "test_results",
      "collection": "power_consumption2"
    }
  },
  "cpu-base-freq": 1900,
  "cpu-error-threshold": 2.0,
  "disable-dram-formula": true,
  "sensor-reports-frequency": 1000
}

Alternative Source or Destination

If you want to use another Source or Destination, please check the documentation here and modify your configuration according to the Source or Destination that you want to use.

Once you have your configuration file, run SmartWatts using one of the following command lines, depending on the installation you use:

DockerPip

docker run -t \
--net=host \
-v $(pwd)/config_file.json:/config_file.json \
powerapi/smartwatts-formula --config-file /config_file.json

python -m smartwatts --config-file config_file.json

Combining the three Running Methods

The three running methods can be used to define a configuration. The priority is as follows:

CLI
Environment Variables
Configuration File

This means that parameters values defined via the CLI have the highest priority while values defined via Environment Variables will be preserved regarding those defined by a Configuration File.