#+TITLE: Estimating the end-to-end energy consumption of low-bandwidth IoT applications for WiFi devices
#+EXPORT_EXCLUDE_TAGS: noexport
#+STARTUP: hideblocks
#+OPTIONS: H:5 author:nil email:nil creator:nil timestamp:nil skip:nil toc:nil ^:nil
#+LATEX_CLASS: llncs
#+LATEX_HEADER: \usepackage{hyperref}
#+LATEX_HEADER: \usepackage{booktabs}
#+LATEX_HEADER: \usepackage{subfigure}
#+LATEX_HEADER: \usepackage{graphicx}
#+LATEX_HEADER: \usepackage{xcolor}
#+LATEX_HEADER: \author{
#+LATEX_HEADER: Loic Guegan and
#+LATEX_HEADER: Anne-Cécile Orgerie\\
#+LATEX_HEADER: }
#+LATEX_HEADER: \institute{Univ Rennes, Inria, CNRS, IRISA, Rennes, France\\
#+LATEX_HEADER: Emails: loic.guegan@irisa.fr, anne-cecile.orgerie@irisa.fr
#+LATEX_HEADER: }
#+BEGIN_EXPORT latex
\newcommand{\hl}[1]{\textcolor{red}{#1}}
#+END_EXPORT
#+BEGIN_EXPORT latex
\begin{abstract}
Information and Communication Technology takes a growing part in the
worldwide energy consumption. One of the root causes of this increase
lies in the multiplication of connected devices. Each object of the
Internet-of-Things often does not consume much energy by itself. Yet,
their number and the infrastructures they require to properly work
have leverage. In this paper, we combine simulations and real
measurements to study the energy impact of IoT devices. In particular,
we analyze the energy consumption of Cloud and telecommunication
infrastructures induced by the utilization of connected devices, And
we propose an end-to-end energy consumption model for these devices.
\end{abstract}
#+END_EXPORT
* Introduction
In 2018, Information and Communication Technology (ICT) was estimated
to absorb around 3% of the global energy consumption
\cite{ShiftProject}. This consumption is estimated to grow at a rate
of 9% per year \cite{ShiftProject}. This alarming growth is explained
by the fast emergence of numerous new applications and new ICT
devices. These devices supply services for smart building, smart
factories and smart cities for instance, providing optimized decisions
based on data produced by smart devices. All these connected devices
constitute the Internet of Things (IoT): connected devices with
sensors producing data, actuators interacting with their environment
and communication means.
This increase in number of devices implies an increase in the energy
needed to manufacture and utilize all these devices. Yet, the overall energy
bill of IoT also comprises indirect costs as it relies on computing and
networking infrastructures that consume energy to enable smart
services. Indeed, IoT devices communicate with Cloud computing
infrastructures to store, analyze and share their data.
In February 2019, a report by Cisco stated that ``IoT connections will
represent more than half (14.6 billion) of all global connected
devices and connections (28.5 billion) by 2022" \cite{Cisco2019}. This
will represent more than 6% of global IP traffic, against 3% in
2017 \cite{Cisco2019}. This increasing impact of IoT devices on
Internet connections induces a growing weight on ICT energy
consumption.
The energy consumption of IoT devices themselves is only the top of
the iceberg: their use induce energy costs in communication and cloud
infrastructures. In this paper, we estimate the overall energy
consumption of an IoT device environment by combining simulations and
real measurements. We focus on a given application with low bandwidth
requirement and we evaluate its overall energy consumption: from the
device, through telecommunication networks, and up to the Cloud data
center hosting the application. From this analysis, we derive an
end-to-end energy consumption model that can be used to assess the
consumption of other IoT devices.
While some IoT devices produce a lot of data, like smart vehicles for
instance, many others generate only a small amount of data, like smart
meters. However, the scale matters here: many small devices can end up
producing big data volumes. As an example, according to a report
published by Sandvine in October 2018, the Google Nest Thermostat is
the most significant IoT device in terms of worldwide connections: it
represents 0.16% of all connections, ranging 55th on the list of
connections \cite{Sandvine2018}. As a comparison, the voice assistants
Alexa and Siri are respectively 97th and 102nd with 0.05% of all
connections \cite{Sandvine2018}. This example highlights the growing
importance of low-bandwidth IoT applications on Internet
infrastructures, and consequently on their energy consumption.
In this paper, we focus on IoT devices for low-bandwidth applications
such as smart meters or smart sensors. These applications send few
data periodically to cloud servers, either to store them or to get
computing power and take decisions. This is a first step towards a
comprehensive characterization of the IoT energy footprint. While few
studies address the energy consumption of high-bandwidth IoT
applications \cite{li_end--end_2018}, to the best of our knowledge,
none of them targets low-bandwidth applications, despite their growing
importance on the Internet infrastructures.
Low-bandwidth IoT applications, such as the Nest Thermostat, often
relies on sensors powered by batteries. For such sensors, reducing
their energy consumption is a critical target. Yet, we argue that
end-to-end energy models are required to estimate the overall impact
of IoT devices and to understand how to reduce their complete energy
consumption. Without such models, one could optimize the consumption
of on-battery devices at a heavier cost for cloud servers and
networking infrastructures, resulting on an higher overall energy
consumption. Using end-to-end models could prevent these unwanted
effects.
Our contributions include:
- a characterization of low-bandwidth IoT applications;
- an analysis of the energy consumption of a low-bandwidth IoT
application including the energy consumption of the IoT device and
the consumption induced by its utilization on the Cloud and
telecommunication infrastructures;
- an end-to-end energy model for low-bandwidth IoT applications.
The paper is organized as follows. Section \ref{sec:sota} presents the
state of the art. The low-bandwidth IoT application is characterized
in Section \ref{sec:usec}, and details on its architecture are
provided in Section \ref{sec:model}. Section \ref{sec:eval} provides
our experimental results using real measurements and
simulations. Section \ref{sec:discuss} discusses the key findings an
the end-to-end energy model. Finally, Section \ref{sec:cl} concludes
this work and presents future work.
* Related Work
#+LaTeX: \label{sec:sota}
** Energy consumption of IoT devices
Smart apps and devices everywhere
Smart industry \cite{Wang2016} : archi with sensing devices, cloud
server, user applications and networks
IoT archi : devices, gateways, fog and clouds \cite{Samie2016}
Smart cities \cite{Ejaz2017}
Smart building \cite{Minoli2017}
home automation, smart agriculture, eHealth, logistics, smart grids
product life-cycle energy management \cite{Tao2016}
focusing on access network technologies \cite{Gray2015},
improving device transmission \cite{Andres2017}
modeling the energy consumption of WSN devices \cite{Martinez2015} or
the WiFi transmission \cite{ns3-energywifi}
on organizing wireless sensor communications to increase the network
lifetime \cite{Wang2016}
CO2 impact of IoT and fog computing architectures vs Cloud
\cite{Sarkar2018}
Fog archi to use more renewable energy \cite{li_end--end_2018} or
reduce communication costs \cite{jalali_fog_2016}
** Energy consumption of network and cloud infrastructures
net models
server models + VM sharing
* Characterization of low-bandwidth IoT applications
#+LaTeX: \label{sec:usec}
** Application Characteristic
#+BEGIN_COMMENT
The IoT part is composed of an Access Point (AP), connected to several sensors using WIFI. In the
system, the IoT part is considered as the part where the system data are created. In fact, the
data life cycle start when the sensors records their respective local temperature at a frequency
$f$ and the local timestamp. Then, these data are transmitted through the network along with an
arbitrary sensor id of 128 bits. Finally, the AP is in charge to transmit the data to the cloud
using the network part.
The network part is considered as the medium that link the IoT part to the cloud. It is composed
of several network switches and router and it is considered to be a wired network.
#+END_COMMENT
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{./plots/home.png}
\caption{Overview of IoT devices.}
\label{fig:IoTdev}
\end{figure}
#+END_EXPORT
** Cloud Infrastructure
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\includegraphics[width=0.85\linewidth]{./plots/parts2.png}
\caption{Overview of the IoT architecture.}
\label{fig:parts}
\end{figure}
#+END_EXPORT
* Experimental setup
\hl{Ajouter \% de bande passante utilisé par les applis low-rate}
#+LaTeX: \label{sec:model}
Our system model is divided in three parts. First, the IoT and the network parts are modeled through
simulations. Then, the Cloud part is modeled using real servers connected to wattmeters. In this way,
it is possible to evaluate the end-to-end energy consumption of the system.
** IoT Part
In the first place, the IoT part is composed of several sensors connected to an Access Point (AP)
which form a cell. This cell is evaluated using the ns-3 network simulator. Consequently, we setup
between 5 and 15 sensors connected to the AP using WiFi 5GHz 802.11n. The node are placed
randomly in a rectangle of $400m^2$ around the AP which corresponds to a typical real use case. All
the cell nodes are setup with the default WIFI energy model provided by ns-3. The different
energy values used by the energy model are provided on Table \ref{tab:wifi-energy}. These energy
were extracted from previous work\cite{halperin_demystifying_nodate,li_end--end_2018} on
802.11n. Besides, we suppose that the energy source of each nodes are unlimited and thus each of
them can communicate until the end of all the simulations.
As a scenario, sensors send 192 bits packets to the AP composed of: \textbf{1)} A 128 bits
sensors id \textbf{2)} A 32 bits integer representing the temperature \textbf{3)} An integer
timestamp representing the temperature sensing time to store them as time series. The data are
transmitted immediately at each sensing interval $I$ varied from 1s to 60s. Finally, the AP is in
charge of relaying data to the cloud via the network part.
#+BEGIN_EXPORT latex
\begin{table}[]
\centering
\caption{Simulations Energy Parameters}
\label{tab:wifi-energy}
\subtable[Wifi]{
\begin{tabular}{@{}lr@{}}
Parameter & Value \\ \midrule
Supply Voltage & 3.3V \\
Tx & 0.38A \\
Rx & 0.313A \\
Idle & 0.273A \\ \bottomrule
\end{tabular}}
\hspace{0.3cm}
\subtable[Network]{
\label{tab:net-energy}
\begin{tabular}{@{}lr@{}}
Parameter & Value \\ \midrule
Idle & 1W \\
Bytes (Tx/Rx) & 3.4nJ \\
Pkt (Tx/Rx) & 192.0nJ \\ \bottomrule
\end{tabular}
}
\end{table}
#+END_EXPORT
** Network Part
The network part represents the a network section starting from the AP to the Cloud excluding the
server. It is also model into ns-3. We consider the server to be 9 hops away from the AP with a
typical round-trip latency of 100ms from the AP to the server. Each node from the AP to the Cloud
is assume to be network switches with static and dynamic network energy consumption. The first 8
hop are edge switches and the last one is consider to be a core switch as mention in
\cite{jalali_fog_2016}. ECOFEN \cite{orgerie_ecofen:_2011} is used to model the energy
consumption of the network part. ECOFEN is a ns-3 network energy module dedicated to wired
network. It is based on an energy-per-bit model including static energy consumption by assuming a
linear relation between the amount of data sent to the network interface and its power
consumption. The different energy values used to instantiate the ECOFEN energy model for the
network part are shown in Table \ref{tab:net-energy} and come from previous work
\cite{cornea_studying_2014-1}.
** Cloud Part
Finally, to measure the energy consumed by the server, we used real server from the large-scale
test-beds Grid5000 (G5K). In fact, G5K has a cluster called Nova composed of several nodes which
are connected to watt-meters. In this way, we can benefit from real energy measurements. The
server used in the experiment include an Intel Xeon E5-2620 processor with 64 GB of RAM and 600GB
of disk space on a Linux based operating system. This server is configured to use KVM as
virtualization mechanism. We deploy a classical Linux x86_64 distribution on the Virtual Machine
(VM) along with a MySQL database. We used different amount of allocated memory for the VM namely
1024MB/2048MB/4096MB to highlight its effects on the server energy consumption.
The sensors requests are simulated using another server. This server is in charge to send hundred
of requests to the VM in order to fill the database. Consequently, it is easy to vary the
different requests characteristics namely: \textbf{1)} The number request, to virtually
add/remove sensors \textbf{2)} The requests interval.
* Evaluation
#+LaTeX: \label{sec:eval}
** IoT/Network Consumption
In a first place, we start by studying the impact of the sensors position on their energy
consumption. To this end, we run several simulations in ns-3 with different sensors position. The
results provided by Table \ref{tab:sensorsSendIntervalEffects} show that sensors position have a very low impact
on the energy consumption and on the application delay. It has an impact of course, but it is very
limited. This due to the fact that in such a scenario with very small number of communications
spread over the time, sensors don't have to contend for accessing to the Wifi channel.
#+BEGIN_EXPORT latex
% Please add the following required packages to your document preamble:
% \usepackage{booktabs}
\begin{table*}[]
\centering
\caption{Sensors send interval effects}
\label{tab:sensorsSendIntervalEffects}
\begin{tabular}{@{}lrrrrr@{}}
\toprule
Sensors Send Interval & 10s & 30s & 50s & 70s & 90s \\ \midrule
Sensors Power Consumption & 13.517\hl{94}W & 13.517\hl{67}W & 13.51767W & 13.51767W & 13.517\hl{61}W \\
Network Power Consumption & 10.441\hl{78}W & 10.441\hl{67}W & 10.44161W & 10.44161W & 10.441\hl{61}W \\
Average Appplication Delay & 17.81360s & 5.91265s & 3.53509s & 2.55086s & 1.93848s \\ \bottomrule
\end{tabular}
\end{table*}
#+END_EXPORT
Previous work \cite{li_end--end_2018} on similar scenario shows that increasing application
accuracy impact strongly the energy consumption in the context of data stream analysis. However,
in our case, application accuracy is driven by the sensing interval and thus, the transmit
frequency of the sensors. Therefore, we varied the transmission interval of the sensors from 1s
to 60s. Some of these results are proposed on Table \ref{tab:sensorsSendIntervalEffects}. In
case of small and sporadic network traffic, these results show that with a reasonable
transmission interval the energy consumption of the IoT/Network if almost not affected by the
variation of this transmission interval. In fact, transmitted data are not large enough to
leverage the energy consumed by the network.
The number of sensors is a dominant factor that leverage the energy consumption of the
IoT/Network part. Therefore, we varied the number of sensors in the Wifi cell to analyze its
impact. The Figure \ref{fig:sensorsNumber} represents the energy consumed by each simulated part
according the the number of sensors. It is clear that the energy consumed by the network is the
dominant part. However, since the number of sensors is increasing the energy consumed by the
network will become negligible face to the energy consume by the sensors. In fact, deploying new
sensors in the cell do not introduce much network load. To this end, sensors energy consumption
is dominant.
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{./plots/numberSensors-WIFINET.png}
\caption{Analysis of the variation of the number of sensors on the IoT/Network part energy consumption.}
\label{fig:sensorsNumber}
\end{figure}
#+END_EXPORT
** Cloud Energy Consumption
In this End-To-End energy consumption study, cloud account for a huge part of the overall energy
consumption. According a report \cite{shehabi_united_2016-1} on United States data center energy
usage, the average Power Usage Effectiveness (PUE) of an hyper-scale data center is 1.2. Thus, in
our analysis, all energy measurement on cloud server will account for this PUE.
In a first place, we analyze the impact of the VM allocated memory on the server energy
consumption. Figure \ref{fig:vmSize} depict the server energy consumption according to the VM
allocated memory for 20 sensors sending data every 10s. Note that horizontal red line represent
the average energy consumption for the considered sample of energy values. We can see that at
each sensing interval, server face to peaks of energy consumption. However, VM allocated memory
do not influence energy consumption. In fact, simple database requests do not need any particular
huge memory access and processing time. Thus, remaining experiments are based on VM with 1024MB
of allocated memory.
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\includegraphics[width=0.8\linewidth]{./plots/vmSize-cloud.png}
\caption{VM size impact on the server energy consumption using 20 sensors sending data every 10s}
\label{fig:vmSize}
\end{figure}
#+END_EXPORT
Next, we study the effects of increasing the number of sensors on the server energy consumption.
Figure \ref{fig:sensorsNumber-server} present the results of the average server energy
consumption when varying the number of sensors from 20 to 500 while Figure
\ref{fig:sensorsNumber-WPS} present the average server energy cost per sensors according to the
number of sensors. These results show a clear linear relation between the number of sensors and
the server energy consumption. Moreover, we can see that the more sensors we have per server, the
more energy we can save. In fact, since the idle server energy consumption is high, it is more
energy efficient to maximize the number of sensors per server. As shown on Figure
\ref{fig:sensorsNumber-WPS}, a significant amount of energy can be save when passing from 20 to
300 sensors per server.
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\subfigure[Average server energy consumption]{
\includegraphics[width=0.4\linewidth]{./plots/sensorsNumberLine-cloud.png}
\label{fig:sensorsNumber-server}
}
\hspace{0.5cm}
\subfigure[Average sensors energy cost on server]{
\includegraphics[width=0.4\linewidth]{./plots/WPS-cloud.png}
\label{fig:sensorsNumber-WPS}
}
\caption{Server energy consumption for sensors sending data every 10s}
\label{fig:sensorsNumber-cloud}
\end{figure}
#+END_EXPORT
A last parameter can leverage server energy consumption namely sensors send interval. In addition
to increasing the application accuracy, sensors send interval increase network traffic and
database accesses. Figure \ref{fig:sensorsFrequency} present the impact on the server energy
consumption of changing the send interval of 50 sensors to 1s, 10s and 30s. We can see that, the
lower sensors send interval is, the more server energy consumption peaks occurs. Therefore, it
leads to an increase of the server energy consumption.
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\includegraphics[scale=0.5]{plots/sendInterval-cloud.png}
\label{fig:sensorsFrequency}
\caption{Server energy consumption for 50 sensors sending request at different interval.}
\end{figure}
#+END_EXPORT
** End-To-End Consumption
To have an overview of the energy consume by the system, it is important to consider the
end-to-end energy consumption. The Figure \ref{fig:end-to-end} represents the end-to-end system
energy consumption while varying the number of sensors. It is important to see that, for
small-scale systems, the server energy consumption is dominant face to the energy consumed by the
sensors. However, since we are using a single server, large-scale sensors deployment lead to an
increasing consumption of energy in the IoT part. On the other side, network energy consumption
is stable regarding to the number of sensors since the system use case do not required large data
transfer. Thus, it is important to remember that, to save energy, we should maximize the number
of sensors handle by each cloud server while keeping reasonable sensors request intervals.
#+BEGIN_EXPORT latex
\begin{figure}
\centering
\hspace{1cm}
\includegraphics[scale=0.3]{plots/final.png}
\label{fig:end-to-end}
\caption{End-to-end network energy consumption using sensors interval of 10s}
\end{figure}
#+END_EXPORT
* Discussion
#+LaTeX: \label{sec:discuss}
* Conclusion
#+LaTeX: \label{sec:cl}
\bibliographystyle{IEEEtran}
\bibliography{references}
* Data Provenance :noexport:
:PROPERTIES:
:header-args: :eval never-export
:END:
** Data Analysis (R Scripts)
*** Utils
**** R
RUtils is intended to load logs (data.csv) and providing
simple plot function for them.
#+NAME: RUtils
#+BEGIN_SRC R :eval never
library("tidyverse")
# Fell free to update the following
labels=c(time="Time (s)",sensorsSendInterval="Sensors Send Interval (s)", sensorsNumber="Number of sensors")
PUE=1.2
ns3SimTime=1800
g5kSimTime=300
loadData=function(path){
data=read_csv(path)
if("sensorsEnergy"%in%colnames(data)){ # If it is ns3 logs
data=data%>%mutate(sensorsEnergy=sensorsEnergy/ns3SimTime) # Convert to watts
data=data%>%mutate(networkEnergy=networkEnergy/ns3SimTime)
data=data%>%mutate(networkEnergy=networkEnergy+getSwitchesIDLE(sensorsNumber,sensorsSendInterval)) # Add Idle conso of switches
data=data%>%mutate(totalEnergy=totalEnergy/ns3SimTime)
}
else{ # Log from g5k
data=data%>%mutate(energy=energy*PUE) # Take into account PUE
data=data%>%filter(time<=g5kSimTime) # Remove extrats values (theorical sim time != real sim time)
}
}
getSwitchesIDLE=function(nbSensors, sendInterval){
pktSize=192
nEdgeRouter=8
nCoreRouter=1
EdgeIdle=4095
EdgeMax=4550
EdgeTraffic=560*10^9
CoreIdle=11070
CoreMax=12300
CoreTraffic=4480*10^9
# Apply 0.6 factor
EdgeTraffic=EdgeTraffic*0.6
CoreTraffic=CoreTraffic*0.6
totalTraffic=pktSize/sendInterval*nbSensors
EdgeConso=EdgeIdle*(totalTraffic/EdgeTraffic)
CoreConso=CoreIdle*(totalTraffic/CoreTraffic)
return(EdgeConso+CoreConso)
}
# Get label according to varName
getLabel=function(varName){
if(is.na(labels[varName])){
return(varName)
}
return(labels[varName])
}
applyTheme=function(plot,...){
palette<- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")
plot=plot+
theme_bw(...)+
scale_fill_manual(values=palette)+
scale_colour_manual(values=palette)
return(plot)
}
#+END_SRC
**** Bash
***** Plots -> PDF
Merge all plots in plots/ folder into a pdf file.
#+NAME: plotToPDF
#+BEGIN_SRC bash :results output :noweb yes
orgFile="plots/plots.org"
<<singleRun>> # To get all default arguments
# Write helper function
function write {
echo "$1" >> $orgFile
}
echo "#+TITLE: Analysis" > $orgFile
write "#+LATEX_HEADER: \usepackage{fullpage}"
write "#+OPTIONS: toc:nil"
# Default arguments
write '\begin{center}'
write '\begin{tabular}{lr}'
write 'Parameters & Values\\'
write '\hline'
write "sensorsPktSize & ${sensorsPktSize} bytes\\\\"
write "sensorsSendInterval & ${sensorsSendInterval}s\\\\"
write "sensorsNumber & ${sensorsNumber}\\\\"
write "nbHop & ${nbHop}\\\\"
write "linksBandwidth & ${linksBandwidth}Mbps\\\\"
write "linksLatency & ${linksLatency}ms\\\\"
write '\end{tabular}'
write '\newline'
write '\end{center}'
for plot in $(find plots/ -type f -name "*.png")
do
write "\includegraphics[width=0.5\linewidth]{$(basename ${plot})}"
done
# Export to pdf
emacs $orgFile --batch -f org-latex-export-to-pdf --kill
#+END_SRC
***** CSVs -> data.csv (G5K)
Merge all energy file into one (and add additional fields).
#+NAME: G5K-mergeCSV
#+BEGIN_SRC sh
#!/bin/bash
whichLog="last"
logsLocation="logs/g5k"
whichLog="${logsLocation}/${whichLog}"
logFile="$(dirname $(readlink -f $0))"/$whichLog/simLogs.txt
dataFile=$(dirname "$logFile")/data.csv
getValue () {
line=$(echo "$1" | grep "Simulation para"|sed "s/Simulation parameters: //g")
key=$2
echo "$line"|awk 'BEGIN{RS=" ";FS=":"}"'$key'"==$1{gsub("\n","",$0);print $2}'
}
##### Add extract info to energy #####
IFS=$'\n'
for cmd in $(cat $logFile|grep "Simulation parameters")
do
nodeName=$(getValue $cmd serverNodeName)
from=$(getValue $cmd simStart)
to=$(getValue $cmd simEnd)
vmSize=$(getValue $cmd vmSize)
nbSensors=$(getValue $cmd nbSensors)
simKey=$(getValue $cmd simKey)
sendInterval=$(getValue $cmd sensorsSendInterval)
csvFile="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}.csv"
csvFileIDLE="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}_IDLE.csv"
tmpFile=${csvFile}_tmp
echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $tmpFile
minTs=$(tail -n+2 $csvFile|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
minTsIDLE=$(tail -n+2 $csvFileIDLE|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
tail -n+2 ${csvFile} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTs'",sim,"'$sendInterval'}' >> $tmpFile
tail -n+2 ${csvFileIDLE} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTsIDLE'",IDLE,"'$sendInterval'}' >> $tmpFile
done
##### Fill dataFile #####
echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $dataFile
for tmpFile in $(find ${whichLog}/*_tmp -type f)
do
tail -n+2 $tmpFile >> $dataFile
rm $tmpFile # Pay attention to this line :D
done
#+END_SRC
#+RESULTS: G5K-mergeCSV
#+RESULTS: mergeCSV
***** Log -> data.csv (ns3)
logToCSV extract usefull data from logs and put them into logs/data.csv.
#+NAME: NS3-logToCSV
#+BEGIN_SRC bash :results output
logsFolder="./logs/ns3/last/"
csvOutput="$logsFolder/data.csv"
# First save csv header line
aLog=$(find $logsFolder -type f -name "*.org"|head -n 1)
metrics=$(cat $aLog|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[1]);if(i<NF)printf(",");else{print("")}}}' > $csvOutput
# Second save all values
for logFile in $(find $logsFolder -type f -name "*.org")
do
metrics=$(cat $logFile|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[2]);if(i<NF)printf(",");else{print("")}}}' >> $csvOutput
done
#+END_SRC
#+RESULTS: NS3-logToCSV
*** Plot Scripts
**** Random R Scripts
Table sensorsSendInterval~Sensors+NetEnergyconsumption
#+BEGIN_SRC R :noweb yes :results output
<<RUtils>>
data=loadData("logs/ns3/last/data.csv")
sensorsE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,sensorsEnergy)%>%arrange(sensorsSendInterval)
delay=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,avgDelay)%>%arrange(sensorsSendInterval)
netE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,networkEnergy)%>%arrange(sensorsSendInterval)
formatData=right_join(sensorsE,netE)%>%right_join(delay)%>%filter(((sensorsSendInterval/10)%%2)!=0)
print(t(formatData))
#+END_SRC
#+RESULTS:
: [,1] [,2] [,3] [,4] [,5]
: sensorsSendInterval 10.00000 30.00000 50.00000 70.00000 90.00000
: sensorsEnergy 13.51794 13.51767 13.51767 13.51767 13.51761
: networkEnergy 10.44178 10.44167 10.44161 10.44161 10.44161
: avgDelay 17.81360 5.91265 3.53509 2.55086 1.93848
Figure Sensors Position ~ Energy/Delay
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsPosition-delayenergy.png
<<RUtils>>
tr=11 # Offset to center delay plot
data=loadData("logs/ns3/last/data.csv")
data=data%>%filter(simKey=="SENSORSPOS",sensorsNumber==9)
p=ggplot(data,aes(y=sensorsEnergy,x=positionSeed,color="Energy"))+xlab(getLabel("Sensors Position Seed"))+ylab(getLabel("Sensors Power Consumption (W)"))+
geom_line()+geom_point()+geom_line(aes(y=(avgDelay-tr),color="Delay"))+geom_point(aes(y=(avgDelay-tr),color="Delay"))+expand_limits(y=c(0,15))+
scale_y_continuous(sec.axis = sec_axis(~.+tr, name = "Application Delay (s)")) +
guides(color=guide_legend(title="Curves"))
p=applyTheme(p)
p=p+theme(axis.title.y.right = element_text(margin = margin(t = 0, r = -8, b = 0, l = 10)))
ggsave("plots/sensorsPosition-delayenergy.png",dpi=80, width=4, height=3.2)
#+END_SRC
#+RESULTS:
[[file:plots/sensorsPosition-delayenergy.png]]
Watt per sensor on server
#+BEGIN_SRC R :noweb yes :results output
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()
data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
data=data%>%distinct(nbSensors,.keep_all=TRUE)
data=data%>%mutate(WPS=(avgEnergy/nbSensors))
print(data%>%select(WPS,nbSensors))
#+END_SRC
Impact of vm size
#+BEGIN_SRC R :results graphics :file plots/vmSizeBar-cloud.png
library("tidyverse")
PUE=1.2
# Load data
data=read_csv("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="vmSize")%>%mutate(energy=PUE*energy)%>%filter(time<=300)
data=data%>%group_by(vmSize)%>%mutate(energy=mean(energy))%>%slice(1L)%>%ungroup()
data=data%>%mutate(vmSize=as.character(vmSize))
ggplot(data) + geom_bar(aes(x=vmSize,y=energy),stat="identity")+expand_limits(y=c(75,100))+ylab("Server Power Consumption (W)")+
xlab("Experiment Time (s)")+scale_y_log10()
ggsave("plots/vmSizeBar-cloud.png",dpi=90,height=3,width=6)
#+END_SRC
#+RESULTS:
[[file:plots/vmSizeBar-cloud.png]]
#+NAME: ssiNet
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-net.png
<<NS3-RUtils>>
# Load Data
data=read_csv("logs/ns3/last/data.csv")
data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=networkEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("networkEnergy"))+
geom_line()+labs(title="For 20 sensors")
ggsave("plots/sensorsSendInterval-net.png",dpi=80)
#+END_SRC
Effect of the number of sensors on the application delay
#+BEGIN_SRC R :noweb yes :results graphics :file plots/delay-nbsensors.png
<<NS3-RUtils>>
# Load Data
data=read_csv("logs/ns3/last/data.csv")
data%>%filter(simKey=="NBSENSORS") %>% ggplot(aes(y=avgDelay,x=sensorsNumber))+xlab(getLabel("sensorsNumber"))+ylab(getLabel("avgDelay"))+
geom_line()+labs(title="For 20 sensors")
ggsave("plots/delay-nbsensors.png",dpi=80)
#+END_SRC
#+RESULTS:
[[file:plots/delay-nbsensors.png]]
#+NAME: ssiWifi
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-wifi.png
<<NS3-RUtils>>
data=read_csv("logs/ns3/last/data.csv")
data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=sensorsEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("sensorsEnergy"))+
geom_line() + geom_line()+labs(title="For 20 sensors")
ggsave("plots/sensorsSendInterval-wifi.png",dpi=80)
#+END_SRC
#+BEGIN_SRC R :results graphics :noweb yes :file plots/plot-final.png :session *R*
<<NS3-RUtils>>
simTime=1800
# Network
data=read_csv("logs/ns3/last/data.csv")
data=data%>%filter(simKey=="NBSENSORS")
dataC5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
dataC10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
dataNet=rbind(dataC5,dataC10)%>%mutate(type="Network")
# Network
dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber)
dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber)
dataS=rbind(dataS5,dataS10)%>%mutate(type="Sensors")
data=rbind(dataNet,dataS)%>%mutate(sensorsNumber=as.character(sensorsNumber))
ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="Part"))
ggsave("plots/plot-final.png",dpi=80)
#+END_SRC
**** Plot In Paper
Power sensors vs network
#+BEGIN_SRC R :noweb yes :results graphics :file plots/numberSensors-WIFINET.png :session *R*
<<RUtils>>
data=loadData("logs/ns3/last/data.csv")
data=data%>%filter(simKey=="NBSENSORS")
dataW=data%>%mutate(energy=sensorsEnergy)%>% mutate(type="Sensors") %>% select(sensorsNumber,energy,type)
dataN=data%>%mutate(energy=networkEnergy)%>% mutate(type="Network") %>% select(sensorsNumber,energy,type)
data=rbind(dataN,dataW)
data=data%>%mutate(sensorsNumber=as.character(sensorsNumber))
data=data%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
data=data%>%filter(sensorsNumber%in%c(2,4,6,8,10))
p=ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),position="dodge",stat="identity")+
xlab(getLabel("sensorsNumber"))+ ylab("Power Consumption (W)") + guides(fill=guide_legend(title=""))
p=applyTheme(p)+theme(text = element_text(size=15))
size=5
ggsave("plots/numberSensors-WIFINET.png",dpi=90,width=size,height=size-1)
#+END_SRC
#+RESULTS:
[[file:plots/numberSensors-WIFINET.png]]
Final plot: Energy cloud, network and sensors
#+BEGIN_SRC R :noweb yes :results graphics :file plots/final.png
<<RUtils>>
# Linear Approx
approx=function(data1, data2,nbSensors){
x1=data1$sensorsNumber
y1=data1$energy
x2=data2$sensorsNumber
y2=data2$energy
a=((y2-y1)/(x2-x1))
b=y1-a*x1
return(a*nbSensors+b)
}
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="nbSensors")
# Cloud
data20=data%>%filter(nbSensors==20)%>%mutate(energy=mean(energy)) %>% slice(1L)
data100=data%>%filter(nbSensors==100)%>%mutate(energy=mean(energy)) %>% slice(1L)
data300=data%>%filter(nbSensors==300)%>%mutate(energy=mean(energy)) %>% slice(1L)
dataCloud=rbind(data20,data100,data300)%>%mutate(sensorsNumber=nbSensors)%>%mutate(type="Cloud")%>%select(sensorsNumber,energy,type)
# Network
data=loadData("./logs/ns3/last/data.csv")
data=data%>%filter(simKey=="NBSENSORS")
dataN5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
dataN10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
dataNet=rbind(dataN5,dataN10)
fakeNet=tibble(sensorsNumber=c(20,100,300))
fakeNet=fakeNet%>%mutate(energy=approx(dataN5,dataN10,sensorsNumber),type="Network")
# Sensors
dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber)
dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber)
dataS=rbind(dataS5,dataS10)
fakeS=tibble(sensorsNumber=c(20,100,300))
fakeS=fakeNet%>%mutate(energy=approx(dataS5,dataS10,sensorsNumber),type="Sensors")
# Combine Net/Sensors/Cloud and order factors
fakeData=rbind(fakeNet,fakeS,dataCloud)
fakeData=fakeData%>%mutate(sensorsNumber=as.character(sensorsNumber))
fakeData=fakeData%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
fakeData$type=factor(fakeData$type,ordered=TRUE,levels=c("Sensors","Network","Cloud"))
# Plot
p=ggplot(fakeData)+geom_bar(position="dodge2",colour="black",aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+
xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="System Part"))
p=applyTheme(p)+theme(text = element_text(size=16))
ggsave("plots/final.png",dpi=90,width=8,height=5.5)
#+END_SRC
#+RESULTS:
[[file:plots/final.png]]
Impact of vm size
#+BEGIN_SRC R :noweb yes :results graphics :noweb yes :file plots/vmSize-cloud.png
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="vmSize")%>%filter(time<=300)
data=data%>%mutate(vmSize=paste0(vmSize," MB"))
data=data%>%group_by(vmSize)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~vmSize)+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
xlab("Experiment Time (s)")
p=applyTheme(p)
ggsave("plots/vmSize-cloud.png",dpi=90,height=3,width=6)
#+END_SRC
#+RESULTS:
[[file:plots/vmSize-cloud.png]]
Impact of sensors number
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsNumber-cloud.png
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()
data=data%>%mutate(nbSensorsSort=nbSensors)
data=data%>%mutate(nbSensors=paste0(nbSensors," Sensors"))
data$nbSensors=fct_reorder(data$nbSensors, data$nbSensorsSort)
data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~nbSensors)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)
p=applyTheme(p)
ggsave("plots/sensorsNumber-cloud.png",dpi=90,height=3,width=6)
#+END_SRC
#+RESULTS:
[[file:plots/sensorsNumber-cloud.png]]
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsNumberLine-cloud.png :session *R:2*
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()
data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
data=data%>%distinct(nbSensors,.keep_all=TRUE)
data=data%>%mutate(WPS=(avgEnergy/nbSensors))
p=ggplot(data,aes(x=nbSensors, y=avgEnergy)) + geom_point() +geom_line()+
xlab(getLabel("sensorsNumber"))+ylab("Average server power consumption (W)")
p=applyTheme(p)+theme(text = element_text(size=14))+ expand_limits(y=108)
ggsave("plots/sensorsNumberLine-cloud.png",dpi=90,height=4.5,width=4)
#+END_SRC
#+RESULTS:
[[file:plots/sensorsNumberLine-cloud.png]]
#+BEGIN_SRC R :noweb yes :results graphics :file plots/WPS-cloud.png :session *R:2*
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()
data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
data=data%>%distinct(nbSensors,.keep_all=TRUE)
data=data%>%mutate(WPS=(avgEnergy/nbSensors))
oldNb=data$nbSensors
data=data%>%mutate(nbSensors=as.character(nbSensors))
data$nbSensors=fct_reorder(data$nbSensors,oldNb)
p=ggplot(data,aes(x=nbSensors, y=WPS)) + geom_bar(stat="identity")+
xlab(getLabel("sensorsNumber"))+ylab("Server power cost per sensors (W)")
p=applyTheme(p)+theme(text = element_text(size=14))+ theme(axis.title.y = element_text(margin = margin(t = 0, r = 8, b = 0, l = 0)))
ggsave("plots/WPS-cloud.png",dpi=90,height=4,width=4)
#+END_SRC
#+RESULTS:
[[file:plots/WPS-cloud.png]]
#+BEGIN_SRC R :noweb yes :results graphics :file plots/sendInterval-cloud.png
<<RUtils>>
# Load data
data=loadData("./logs/g5k/last/data.csv")
data=data%>%filter(state=="sim",simKey=="sendInterval")%>%ungroup()
oldSendInterval=data$sendInterval
data=data%>%mutate(sendInterval=paste0(sendInterval,"s"))
data$sendInterval=fct_reorder(data$sendInterval,oldSendInterval)
data=data%>%group_by(sendInterval)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
print(data)
p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~sendInterval)+expand_limits(y=c(0,40))+ylab("Server power consumption (W)")+
xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)
p=applyTheme(p)
ggsave("plots/sendInterval-cloud.png",dpi=120,height=3,width=6)
#+END_SRC
#+RESULTS:
[[file:plots/sendInterval-cloud.png]]
* Emacs settings :noexport:
# Local Variables:
# eval: (unless (boundp 'org-latex-classes) (setq org-latex-classes nil))
# eval: (add-to-list 'org-latex-classes
# '("llncs" "\\documentclass[conference]{llncs}\n \[NO-DEFAULT-PACKAGES]\n \[EXTRA]\n" ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}") ("\\subsubsection{%s}" . "\\subsubsection*{%s}") ("\\paragraph{%s}" . "\\paragraph*{%s}") ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
# End: