Powerapi

The goal of this projet is to provide a set of tools to go forward a greener computing. The idea is to provide software-defined power meters to mesure the power consumption of program. The core of this project is the PowerAPI toolkit for building such power meters.

Getting started

If you want to monitor the energy consumption of your process we have some ready-to-use tools

  • RAPL formula for monitoring the energy consumption of your device.
  • SmartWatts formula for monitoring the energy consumption of your process.
  • Jouleit for mesuring the energy consumption of a program.

How it works

A power meter is an application build with the PowerAPI components that can measure the power consumption of software running on a single machine or on a cluster of machine.

This section present each PowerAPI component type and how connect them to build a power meter.

Power meter Architecture

A power meter is a set of two components, a sensor and a formula, used to produce an estimation of the power consumption of a monitored software.

The sensor collects raw data correlated with the power consumption of the software. The formula is a computational module that use the collected data to determine power consumption. Both of them are connected by a database that is used to transfer information. The global architecture of a power meter is represented bellow.

Powerapi

The two next sub-sections present how a sensor and a formula work and how they should be used.

Sensor

A sensor is an independent software that collects raw data correlated with the power consumption of monitored software.

Data are collected by querying the hardware’s machine that hosts the monitored software. The sensor must be executed on the same machine as the monitored software. The data are collected throughout the duration of the software. For this reason, the sensor must operate in parallel.

Collected data are stored in an external database to make the data available to the formula. This database may be hosted on an other machine.

Usage

Because they collect from different hardware, each sensor are very different from one another. Refer to each sensor documentation to know how to use them.

Formula

A formula is an independent software that compute an estimation of the power consumption of monitored software from the data collected by the sensor.

Sensor Connection

A formula communicate with the sensor via a database (e.g MongoDB). The sensor writes the collected data to the database and the formula reads it afterward.

There are two connection modes:

  • stream mode where the formula read the data from the sensor as they are produced.

  • post-mortem mode which analyses the data already collected by the sensor in a past monitoring phase.

Mailing list

You can follow the latest news and asks questions by subscribing to our mailing list.

Contributing

If you would like to contribute code you can do so through our Github repository by forking it and sending a pull request.

You should start by reading the contribution guide

Publications

Credits and Licence

PowerAPI is an open-source project developed by the Spirals research group (University of Lille and Inria)

This software is licensed under the BSD 3-Clause License, quoted below.

BSD 3-Clause License

Copyright (c) 2021, INRIA Copyright (c) 2021, University of Lille All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

  • Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Installing

RAPL Formula

RAPL Formula is made for tracking the energy consumption of your machine. To install RAPL Formula run the following script in a Terminal.

The script explains what it will do and then pauses before it does it.

Jouleit

Jouleit is made for tracking the energy consumption of a program. Jouleit need gawk to run. You can get the script from the github repository Start jouleit by using ./jouleit.sh cmd.

Smartwatts

Smartwatts is made for tracking the energy consumption of processes on a machine. To install smartwatts run the following script in a Terminal.

The script explains what it will do and then pauses before it does it.

Reports

PowerAPI tools tend to be modular, any sensor could be plugged to any monitoring tool as long as the needed information is provided. We fixed the way of encoding the information. Those encoding are called reports.

A report specify the json fields that should be provided to pass information of a certain kind.

The type of reports required by each formula are specified in their user guide.

All report have a common basis:

  • timestamp : at the format "year-month-dayThour:minutes:secondes". The timestamp should reflect the time at which the information correspond, not the time the information was computed. For example if a power consumption of a CPU is mesured at time t and used to determine the power comsumption of a cgroup in a PowerReport, this report should have timestamp t.

    • target : The target should be the subject of the mesure. For example if you produce a report that contain information relative to a program,domain, etc. The target should refer to it.

  • sensor: It's a name field that should be use to regroup reports for computing modules. For example, in VirtualWatts, the computationnal module need to receive a ProcfsReport and a PowerReport, for both reports to be send to the same computational module they have to have the same sensor.

    A report have the following format:

    "timestamp":$timestamp,
"target":$target,
"sensor":$sensor,
$report_specific_fields

In the following parts we specify the $report_specific_fields for each type of report.

HWPC Report

A HWPCReport should be used to report perfomance counters and RAPL. It specific fields are the following :

  • groups: A list of subreport that can be of three kind, rapl, core and msr.

    Each are presented the same way

    {
  $type: {
    $core_number : {
      $socket_number : {
        List of counter and their value
      }
    }
  }
}

We provide an example of HWPCReport:

{
  "timestamp": "2021-01-13T09:51:22.630",
  "sensor": "sensor_test",
  "target": "influxdb",
  "groups": {
    "core": {
      "0": {
        "0": {
          "CPU_CLK_THREAD_UNHALTED:THREAD_P": 75510,
          "CPU_CLK_THREAD_UNHALTED:REF_P": 2271,
          "time_enabled": 167403,
          "time_running": 167403,
          "LLC_MISSES": 1077,
          "INSTRUCTIONS_RETIRED": 31693
        },
        "1": {
          "CPU_CLK_THREAD_UNHALTED:THREAD_P": 43801,
          "CPU_CLK_THREAD_UNHALTED:REF_P": 1318,
          "time_enabled": 99324,
          "time_running": 99324,
          "LLC_MISSES": 750,
          "INSTRUCTIONS_RETIRED": 15011
        }
      }
    }
  }
}
{
  "timestamp" : "2021-01-13T09:51:22.630",
  "sensor" : "sensor_test",
  "target" : "all",
  "groups" : {
    "rapl" : {
      "0" : {
        "1" : {
			      "RAPL_ENERGY_PKG" : 5709496320,
			      "time_enabled" : 1006717449,
			      "time_running" : 1006717449
          }
      }
    },
    "msr" : {
      "0" : {
        "0" : {
          "MPERF" : 29646849,
	        "APERF" : 12319312,
	        "TSC" : 2122153094,
	        "time_enabled" : 1006580601,
	        "time_running" : 1006580601
        },
        "1" : {
          "MPERF" : 20587012,
          "APERF" : 19838920,
          "TSC" : 2122185970,
	        "time_enabled" : 1006560540,
	        "time_running" : 1006560540
        }
      }
    }
  }
}

Power Report

A PowerReport should be used to transfer information about power consumption. It specific fields are the following :

  • power: A power value in Watts

We provide an example of PowerReport:

{
  "timestamp": "2021-09-14T12:37:37.168817",
  "sensor": "formula_group",
  "target": "all",
  "power": 42
}

Procfs Report

A ProcfsReport should be used to transfer information about cpu usage of process. It specific fields are the following :

  • global_cpu_usage : The used percentage of the cpu.
  • usage: A list of the monitored process with their percentage of cpu usage.

We provide an example of ProcfsReport:

{
  "timestamp": "2021-09-14T12:37:37.168817",
  "sensor": "formula_group",
  "target": ["firefox_cgroup", "emacs_cgroup", "zsh_cgroup", "mongo_cgroup"],
  "usage": {
    "firefox_cgroup": 8.36,
    "emacs_cgroup": 5.52,
    "zsh_cgroup": 0.01,
    "mongo_cgroup": 0.64
  },
  "global_cpu_usage": 27.610000000000014
}

Configuration File

PowerAPI can read configuration through config file or through the CLI. Here we are using config file because they are more readable.

PowerAPI can be configured with the following parameters :

  • verbose (bool) : verbose or quiet mode.
  • stream (bool) : to specify if the reports are provided one by one or are stored in a database.
  • sensor-report-sampling-interval (int): If in stream mode, the time in milliseconds between two reports.
  • input (database) : The database used in input. The way to write their configuration is specified here. Multiple database can be used as inputs.
  • output (database) : The database used in output. The way to write their configuration is specified here. Multiple database can be used as outputs.

All configuration files of PowerAPI based formula follow the next template:

{
  "verbose": $bool,
  "stream": $bool ,
  "sensor-report-sampling-interval" : $int,
"input": {
    $puller_name: {
      "model": $type_of_report,
      "type": $type_of_database,
      $database_parameters
    }
    ... #(Multiple puller can be used)
  },
  "output": {
    $pusher_name: {
      "type": $type_of_database
      $database_parameters
    }
    ... #(Multiple pusher can be used)

  },
  $formula_parameters
}

Visualizing process power consumptions on Grafana

Introduction

This tutorial presents how to visualize the power estimation computed by a formula on a grafana dashboard to obtain this kind of visualisation:

datasource_section

This screenshot show the visualisation of power consumption of a web browser and tools used for monitoring (database, sensor, formula)

In this tutorial, we describe how to connect a formula to a grafana instance, using InfluxDB. Then, we will see how to configure grafana to visualize the power estimation computed by the formula.

This tutorial assume that you know how launch a formula and a sensor to compute power estimation and that you have an InfluxDB and a grafana instance running on your local machine. The InfluxDB instance listen on port 1234 and Grafana instance listen on port 4321.

Connect Grafana to the InfluxDB instance

Connect to your Grafana instance and go to the "Data sources" section (in the configuration part of the side bar).

datasource_section

Click on the "Add data source" button and select "InfluxDB".

Enter a data source name (here we choose "InfluxDB-1"), the instance URI (http://localhost:4321) and the database name you use in the file config_formula.json then click on the "Save and test" button.

datasource_section

Visualize the power consumption on a dashboard in real-time

Go to the "Create Dashboard" section on the side bar to create a new dashboard.

datasource_section

Click on the "add Query" button and write the following query on the Query field to request the power estimations from the InfluxDB measurement: SELECT power FROM "power_consumption" GROUP BY target

Then write $tag_target on the ALIAS BY field to label each graph with the target name

datasource_section

To display the power consumption in real time, you can update the range of the visualisation to last 5 minutes and the refresh dashboard parameter to 5s. This parameter are on the top-right corner of the UI.

datasource_section

How to create a cgroup

If you want to monitor a specific process (or group of process), you have to create a cgroup with perf_event controller and add the process you want to monitor to this cgroup

You can easly do this with the following command :

  • to create the cgroup : cgcreate -g perf_event:new_cgroup_name

with new_cgroup_name the name of the cgroup you want to create (this will be the name wich will identify power consumption of you process)

  • to add your process to the newly created cgroup : cgclassify -g perf_event:new_cgroup_name PID

with PID, the pid of the process you want to monitor. If you want to monitor a program composed of many process, replace PID with $(pidof program_name)

What to use

We provide a set of ready-to-use tools for diverse needs. We list bellow the available tools and the main use cases.

  • RAPL Formula : Mesure of power consumption of each socket.
  • SmartWatts Formula : Mesure of power consumption of cgroup (do not work in VM)
  • VirtualWatts : Mesure of power consumption of cgroup inside a VM
  • Jouleit: Mesure the energy consumed by your program
  • Joule Hunter : Mesure energy consumption of each part of your program.
  • PyJoules : Monitor the energy consumed by specific device of the host machine

If you did not find the tools that respond to your needs you should consider writing it. We provide a tutorial on how to write a formula here.

RAPL

RAPL stand for Running Average Power Limit, it is a feature of recent intel processors that provide the energy consumption of the processor.

The RAPL formula is designed to mesure power consumption of domains (cpu or ram) in real time.

The RAPL formula take HWPC report with RAPL event for each domains. It then return the mesured power in a PowerReport for each domain. This formula do not perform any other computation as its goal is only to track global power consumption in a more readable way than raw RAPL.

Installation

You can use the following script to install RAPL Formula and HWPC Sensor.

From pypi

pip install rapl-formula

From docker

docker pull powerapi/rapl-formula

From deb file

Download the .deb files (rapl-formula and thespian) from the latest release

Install rapl-formula with sudo apt install ./python3-rapl-formula_0.5.0-1_all.deb

Quickstart

For running the RAPL formula we'll need two things:

  • A sensor that provide HWPCReport. We'll use HWPC sensor.
  • A configuration for the formula. We provide an example bellow.
{
  "verbose": true,
  "stream": true,
  "input": {
    "puller": {
      "model": "HWPCReport",
      "type": "mongodb",
      "uri": "mongodb://127.0.0.1",
      "db": "test",
      "collection": "prep"
    }
  },
  "output": {
    "pusher_power": {
      "type": "mongodb",
      "model": "PowerReport",
      "type": "mongodb",
      "uri": "mongodb://127.0.0.1",
      "db": "test",
      "collection": "results"
    }
  },
  "enable-dram-formula": false,
  "sensor-report-sampling-interval": 500
}

Start by installing the hwpc-sensor (see here) and start it (see here). You also need to start an mongodb via the command line docker run -d --name mongo_rapl -p 27017:27017 mongo.

Then run rapl_formula using one of the following command line, depending on the installation you used:

  • via pip : python -m rapl_formula --config-file config_file.json
  • via docker docker run -t --net=host -v $(pwd)/config_file.json:/config_file.json powerapi/rapl-formula --config-file /config_file.json
  • via deb file : rapl-formula --config-file config_file.json

Your power report will be provided in the influxdb. You can watch them in a grafana using the following tutorial

User guide

Configuration parameters

  • enable-cpu-formula (bool) : Enable CPU formula, default=True
  • enable-dram-formula(bool) : Enable DRAM formula, default=True

SmartWatts

SmartWatts is a software-defined power meter based on the PowerAPI toolkit. SmartWatts is a configurable software that can estimate the power consumption of software in real-time. SmartWatts need to receive several metrics provided by hwpc-sensor :

  • The Running Average Power Limit (RAPL)
  • TSC
  • APERF
  • MPERF
  • CPU_CLK_THREAD_UNHALTED:REF_P
  • CPU_CLK_THREAD_UNHALTED:THREAD_P
  • LLC_MISSES
  • INSTRUCTIONS_RETIRED

These metrics are then used as inputs for a power model that estimate the power consumption of each software. The model can derive from the reality, each time the cpu-error-threshold is reached it learn a new power model, using the previous reports.

The choice of those specific metrics is motivated in SmartWatts: Self-Calibrating Software-Defined Power Meter for Containers

Installation

You can use the following script to install Smartwatts and HWPC Sensor.

From pypi

pip install smartwatts

From docker

docker pull powerapi/smartwatts-formula

From deb file

Download the .deb file from the latest release

Install smartwatts with sudo apt install ./python3-smartwatts_0.8.0-1_all.deb

Quickstart

For running the Smartwatts formula we'll need two things:

  • A sensor that provide HWPCReport. We'll use HWPC sensor.
  • A configuration for the formula. We provide an example bellow.
{
  "verbose": true,
  "stream": true,
  "input": {
    "puller": {
      "model": "HWPCReport",
      "type": "mongodb",
      "uri": "mongodb://127.0.0.1",
      "db": "test",
      "collection": "prep"
    }
  },
  "output": {
    "pusher_power": {
      "type": "mongodb",
      "uri": "mongodb://127.0.0.1",
      "db": "test",
      "collection": "prep"
    }
  },
  "cpu-frequency-base": 19,
  "cpu-frequency-min": 4,
  "cpu-frequency-max": 42,
  "cpu-error-threshold": 2.0,
  "disable-dram-formula": true,
  "sensor-report-sampling-interval": 1000
}

The configuration can depend of your hardware, we provide an auto-configuration script.

Start by installing the hwpc-sensor (see here) and start it (see here). You also need to start an mongodb via the command line docker run -d --name mongo_sw -p 27017:27017 mongo.

Then run smartwatts using one of the following command line, depending on the installation you used:

  • via pip : python -m smartwatts --config-file config_file.json
  • via docker docker run -t --net=host -v $(pwd)/config_file.json:/config_file.json powerapi/smartwatts-formula --config-file /config_file.json
  • via deb file : smartwatts --config-file config_file.json

Your power report will be provided in the influxdb. You can watch them in a grafana using the following tutorial

Auto-config Script

This script detect the frequency of your cpu and use them to provide a configuration file for Smartwatts.

#!/usr/bin/env bash

maxfrequency=$(lscpu -b -p=MAXMHZ | tail -n -1| cut -d , -f 1)
minfrequency=$(lscpu -b -p=MINMHZ | tail -n -1 | cut -d , -f 1)
basefrequency=$(lscpu | grep "Model name" | cut -d @ -f 2 | cut -d G -f 1)
basefrequency=$(expr ${basefrequency}\*1000 | bc | cut -d . -f 1)

echo "
{
  \"verbose\": true,
  \"stream\": true,
  \"input\": {
    \"puller\": {
      \"model\": \"HWPCReport\",
      \"type\": \"socket\",
      \"uri\": \"127.0.0.1\",
      \"port\": 8080,
      \"collection\": \"test_hwpc\"
    }
  },
  \"output\": {
    \"pusher_power\": {
      \"type\": \"influxdb\",
      \"model\": \"PowerReport\",
      \"uri\": \"127.0.0.1\",
      \"port\": 8086,
      \"db\": \"test\",
      \"collection\": \"prep\"
    }
  },
  \"cpu-frequency-base\": $basefrequency,
  \"cpu-frequency-min\": $minfrequency,
  \"cpu-frequency-max\": $maxfrequency,
  \"cpu-error-threshold\": 2.0,
  \"disable-dram-formula\": true,
  \"sensor-report-sampling-interval\": 1000
}
" > ./config_file.json

Jouleit

Jouleit is a script that can be used to monitor energy consumption for any program.

Jouleit uses the Intel "Running Average Power Limit" (RAPL) technology that estimates power consumption of the CPU, ram and integrated GPU. This technology is available on Intel CPU since the Sandy Bridge generation(2010).

Installation

Jouleit need gawk to run. You can get the script from the github repository Start jouleit by using ./jouleit.sh cmd.

Flags and options

FlagDescriptionDefault value
-aprint the details of all sockets instead of the aggregationFalse
-bprint the results in the format of *KEY1;VALUE1;KEY2;VALUE2..False
-llist all the available domaines ( CPU, DRAM ..etc ) and print them in the form of a header of csv
-cPrint only the values in csv format ( value1;value2;value3), We recommend using this after running the jouleit with -l Flag to see the order of the measured valuesfalse
-s #Nmeasure only the energy of the socket #Nall the available sockets
-o filenameredirect the output and the log of the executed program in the file `filenamecurrent terminal
-n NRun the programm N times and record the measured values in data1234.csv file

VirtualWatts

VirtualWatts is a software-defined power meter based on the PowerAPI toolkit. VirtualWatts is a configurable software that can estimate the power consumption of software inside a Virtual Machine (VM) in real-time.

VirtualWatts use the estimation of energy consumption of the VM, computed by SmartWatts, and the cpu usage of each process inside the VM, computed by Procfs Sensor.

Installation

You can use the following script to install VirtualWatts and Procfs Sensor.

From pypi

pip install virtualwatts

From docker

docker pull powerapi/virtualwatts-formula

From deb file

Download the .deb file from the latest release

Install virtualwatts with sudo apt install ./python3-virtualtwatts_0.1.1-1_all.deb

Quickstart

For running the VirtualWatts formula we'll need several things:

  • Smartwatts running on the host machine
  • A file SW_output, shared with the host that will be used as database
  • The Procfs sensor running inside the VM
  • A configuration file for each.

VirtualWatts configuration :

{
    'verbose': True,
    "stream":True,
    "input": {
        "puller_filedb": {
            "type": "filedb",
            "model": "PowerReport",
            "filename": "SW_output"
            },
        "puller_tcpdb": {
            "type" : "socket",
            "model": "ProcfsReport",
            "uri": "127.0.0.1",
            "port": self.port
        }
    },
    "output": {
        "power_pusher": {
        "type": "influxdb",
        "model": "PowerReport",
        "uri": "127.0.0.1",
        "port": 8086,
        "db": "test",
        "collection": "prep"
        }
    },
    "delay-threshold": 500,
    "sensor-reports-sampling-interval": datetime.timedelta(500)
}

Start by running Smartwatts with a filedb output. Then run virtualwatts using one of the following command line, depending on the installation you used:

  • via pip : python -m virtualwatts --config-file config_file.json
  • via docker docker run -t --net=host -v $(pwd)/config_file.json:/config_file.json powerapi/virtualwatts --config-file /config_file.json
  • via deb file : virtualwatts --config-file config_file.json

After that run the procfs sensor. Your power report will be provided in the influxdb. You can watch them in a grafana using the following tutorial

Hwpc-Sensor

HardWare Performance Counter (HWPC) Sensor is a tool that monitor the Intel CPU performance counter and the power consumption of CPU.

Hwpc-sensor use the RAPL (Running Average Power Limit) technology to monitor CPU power consumption. This technology is only available on Intel Sandy Bridge architecture or higher.

The sensor use the perf API of the Linux kernel. It is only available on Linux and need to have root access to be used.

The sensor couldn’t be used in a virtual machine, it must access (via Linux kernel API) to the real CPU register to read performance counter values.

Installation

From docker

docker pull powerapi/hwpc-sensor

From deb file

Download the .deb file from the latest release

Install the sensor with sudo apt install hwpc-sensor-1.1.0.deb

Using the binary

You can use the compiled version of the sensor (available here)

Quickstart

For running the sensor, first we need a configuration. We provide an example bellow.

{
  "name": "sensor",
  "verbose": true,
  "frequency": 500,
  "output": {
    "type": "mongodb",
    "uri": "mongodb://127.0.0.1",
    "database": "db_sensor",
    "collection": "report_0"
  },
  "system": {
    "rapl": {
      "events": ["RAPL_ENERGY_PKG"],
      "monitoring_type": "MONITOR_ONE_CPU_PER_SOCKET"
    },
    "msr": {
      "events": ["TSC", "APERF", "MPERF"]
    }
  },
  "container": {
    "core": {
      "events": [
        "CPU_CLK_THREAD_UNHALTED:REF_P",
        "CPU_CLK_THREAD_UNHALTED:THREAD_P",
        "LLC_MISSES",
        "INSTRUCTIONS_RETIRED"
      ]
    }
  }
}

Start a mongo db and then you can start the sensor with the following command line, depending on your installation :

  • from docker : docker run --rm --net=host --privileged --pid=host -v /sys:/sys -v /var/lib/docker/containers:/var/lib/docker/containers:ro -v /tmp/powerapi-sensor-reporting:/reporting -v $(pwd):/srv -v $(pwd)/config_file.json:/config_file.json powerapi/hwpc-sensor --config-file /config_file.json
  • from binary : ./hwpc-sensor --config-file config_file.json

The reports will be provided in your mongodb.

Procfs-Sensor

The Proc Filesystem Sensor is a tool that monitor the CPU usage of cgroup via the linux's proc filesystem. It use pidstat to retreive the percentage of CPU usage of each process. It then use the /sys/fs/perf_event directory to find the appartenance of processes to cgroup.

The sensor need the cgroup version 1. The version 2 is not supported yet.

Installation

From pypi

pip install procfs-sensor

From docker

docker pull powerapi/procfs-sensor

From deb file

Download the .deb file from the latest release

Install the sensor with sudo apt install procfs-sensor-1.1.0.deb

Using the binary

You can use the compiled version of the sensor (available here)

Quickstart

For running the sensor, first you need to define the cgroup you want to track.

The for running the sensor you need configuration. We provide an example bellow.

{
  "name": "procfs_sensor",
  "verbose": true,
  "frequency": 500,
  "output": {
    "type": "socket",
    "uri": "127.0.0.1",
    "port": 8080
  },
  "target": ["cgroup1", "cgroup2", ...]
}

You can start the sensor with the following command line, depending on your installation :

  • from docker : ``docker -v ./config_file.json:/config_file.json run procfs-sensor --config_file config_file.json`
  • from pip : python -m procfs_sensor --config-file config_file.json

You can use a TCP server to retreive the reports.

Database

A database can be used as an input or an output of powerapi formula

For each database the parameters to specify are differents. For each database, its parameters are specified in its section.

MongoDB

If you want to use a mongo database in your formula your have to specify mongodb as the type of a puller or a pusher.

The list of parameters you have to provide :

  • uri : The ip address of your mongo database
  • db: the name of your database
  • collection: the name of the collection inside the database

The default model of mongodb is HWPCReport if used as an input and PowerReport if used as an output.

We provide an example of configuration file.

{
  "type": "mongodb",
  "uri": "mongodb://127.0.0.1",
  "db": "test",
  "collection": "prep"
}

Influx DB 1.8

If you want to use influxdb 1.8 in your formula your have to specify influxedb as the type of a pusher.

The list of parameters you have to provide :

  • uri : IP address of the server with the database
  • port: port of communication
  • db : name of the database

InfluxDB can only be used as an output and its default model is PowerReport.

We provide an example of configuration file :

{
  "tags": "socket",
  "model": "PowerReport",
  "type": "influxdb",
  "uri": "127.0.0.1",
  "port": 8086,
  "db": "test_influxdb"
}

Influx DB 2

If you want to use influxdb 2 in your formula your have to specify influxedb2 as the type of a pusher.

The list of parameters you have to provide :

  • uri : IP address of the server with the database
  • port: port of communication
  • db : name of the bucket (database)
  • org : name of the organisation associated to the bucket
  • token : token for accesing the database. The token ower must have write/read permissions on the database

InfluxDB2 can only be used as an output and its default model is PowerReport.

We provide an example of configuration file :

{
  "model": "PowerReport",
  "type": "influxdb2",
  "uri": "http://127.0.0.1",
  "port": 8086,
  "db": "influxdb2",
  "org": "org_test",
  "token": "mytoken"
}

CSV

If you want to use a csv file in your formula your have to specify csv as the type of a puller or a pusher.

The list of parameters you have to provide :

  • uri : The csv file name

The default model of csv is HWPCReport.

The default model of csv is HWPCReport if used as an input and PowerReport if used as an output.

We provide an example of configuration file.

{
  "type": "csv",
  "uri": "/tmp/sensor_output/"
}

Socket

If you want to use a tcp socket in your formula your have to specify socket as the type of a puller or a pusher. This database is made for stream mode.

The list of parameters you have to provide :

  • uri : The IP address of the server
  • port: The port to use to communicate with the server

The socket can only be used as an input and its default model is HWPCReport.

We provide an example of configuration file.

{
  "type": "socket",
  "uri": "127.0.0.1",
  "port": 8080
}

File Database

If you want to use a file as database in your formula your have to specify filedb as the type of a puller or a pusher. The database is made for stream mode. It should contain only the last report.

The list of parameters you have to provide :

  • filename : The name of the file

This database can only be used as an input and its default model is PowerReport.

We provide an example of configuration file.

{
  "type": "socket",
  "filename": /tmp/database
}

Prometheus

If you want to use a prometheus to expose reports to be scraped, you have to specify prom as the type of a pusher in your formula configuration file.

The list of parameters you have to provide :

  • tags : specify report tags
  • uri : Server ip address
  • port : Server port
  • metric_name : The metric name
  • metric_description : The metric description. Its default value is energy consumption

Promtheus can only be used as an output that monitors reports but they will be not stored by this service. It has to be configured (via static_configs) by using the uri and port defined by the formula configuration file.

We provide an example of configuration file.

{
  "type": "prom",
  "uri": "127.0.0.1",
  "port": 8080,
  "metric_name": test
}
Powerapi

PowerAPI is a middleware toolkit for building software-defined power meters. Software-defined power meters are configurable software libraries that can estimate the power consumption of software in real-time. As a middleware toolkit, PowerAPI offers the capability of assembling power meters «à la carte» to accommodate user requirements. If you want to build your own power meter using PowerAPI, we provide a step by step guide.

Creating your own formula

To develop a formula you have to consider the following parts :

  • The configuration parameters needed by the actor
  • The dispatching rule of the reports to the different formula actors
  • The behavior of the formula actor

We provide a dummy formula that contain the template to develop your own formula. Next we provide a guide on developing your own formula using that template.

Starting

To develop a formula there is two major tasks. How to handle the formula actor, and the behavior of the formula actor.

The first thing you need to decide is what will your formula receive as inputs, how it will treat it and what will it return. Once you've decide this part you need to extract the parameters needed by the formula actor. In the next of this guide we will treat the handling of the formula actor then we will provide some advices on how to develop your formula actor.

Handling the actor

Once you've decided the parameters the formula actor will need, we need to build the chain to pass them from the configuration file to the actor.

First we need to build the parser that will extract the parameters from the configuration file. For that part we are working on generate_dummy_parser in __main__.py. For each parameters arg you should add the following lines :

    parser.add_argument(
        "arg",  # name of the arg
        help="help on arg",  # help to display when --help is used
        type=None,  # type expected
        default=None,  # Default value if not specified
    )

Once the configuration is retrieved from the configuration file it is necessary to check it. For that we use a DummyConfigValidator that will check that each mandatory arguments are present and that each arguments have the correct type.

At this moment the configuration is a Dictionary, it will be transformed to a DummyFormulaConfig (in context.py) to be passed to the actor. You should put each parameters needed in the formula actor as attribute of that class.

In addition of the configuration you need to specify the dispatch rule of route_table. This rule decide which report is send to which formula actor. If you choose SENSOR, reports with the same sensor field will be send to the same actor. If you choose SOCKET the sort will be made on the sensor field.

Your formula

Now that you actor will receive its parameters, you need to build it. Your actor will be initialized two times, one as an object and one as an actor. You first need to define all the attributes as None is the __init__. The supervisor will then call _initialization with a StartMessage containing the configuration.

The receiveMsg_Report(self,message) method will be used each time a report is send to the formula actor. Once you have treated your reports and produced the output reports, send them to the pushers using the following commands :

for name, pusher in self.pushers.items():
    self.log_debug('send ' + str(report) + ' to ' + name)
    self.send(pusher, report)