paper-lowrate-iot/2019-Mascots.org

#+TITLE: Estimating the end-to-end energy consumption of IoT devices along with their impact on Cloud and telecommunication infrastructures

#+EXPORT_EXCLUDE_TAGS: noexport
#+STARTUP: hideblocks
#+OPTIONS: H:5 author:nil email:nil creator:nil timestamp:nil skip:nil toc:nil ^:nil
#+LATEX_CLASS: IEEEtran
#+LATEX_HEADER: \usepackage{hyperref}
#+LATEX_HEADER: \usepackage{booktabs}
#+LATEX_HEADER: \usepackage{subfigure}
#+LATEX_HEADER: \usepackage{graphicx}
#+LATEX_HEADER: \usepackage{xcolor}
#+LATEX_HEADER: \IEEEoverridecommandlockouts 
#+LATEX_HEADER: \author{\IEEEauthorblockN{1\textsuperscript{st} Anne-Cécile Orgerie}
#+LATEX_HEADER: \IEEEauthorblockA{\textit{Univ Rennes, Inria, CNRS, IRISA, Rennes, France} \\
#+LATEX_HEADER: Rennes, France \\
#+LATEX_HEADER: anne-cecile.orgerie@irisa.fr}
#+LATEX_HEADER: \and
#+LATEX_HEADER: \IEEEauthorblockN{2\textsuperscript{nd} Loic Guegan}
#+LATEX_HEADER: \IEEEauthorblockA{\textit{Univ Rennes, Inria, CNRS, IRISA, Rennes, France} \\
#+LATEX_HEADER: Rennes, France \\
#+LATEX_HEADER: loic.guegan@irisa.fr}
#+LATEX_HEADER: }

#+BEGIN_EXPORT latex
\newcommand{\hl}[1]{\textcolor{red}{#1}}
#+END_EXPORT

#+BEGIN_EXPORT latex
\begin{abstract}
Information and Communication Technology takes a growing part in the worldwide energy consumption. One of the root causes of this increase lies in the multiplication of connected devices. Each object of the Internet-of-Things often does not consume much energy by itself. Yet, their number and the infrastructures they require to properly work have leverage. In this paper, we combine simulations and real measurements to study the energy impact of IoT devices. In particular, we analyze the energy consumption of Cloud and telecommunication infrastructures induced by the utilization of connected devices, and we propose an end-to-end energy consumption model for these devices. 
\end{abstract}

\begin{IEEEkeywords}
component, formatting, style, styling, insert
\end{IEEEkeywords}
#+END_EXPORT


* Introduction [2 col]
* Related Work [1 col]
* Use-Case [1 col]
** Application Characteristic

   #+BEGIN_COMMENT
   The IoT part is composed of an Access Point (AP), connected to several sensors using WIFI. In the
   system, the IoT part is considered as the part where the system data are created. In fact, the
   data life cycle start when the sensors records their respective local temperature at a frequency
   $f$ and the local timestamp. Then, these data are transmitted through the network along with an
   arbitrary sensor id of 128 bits. Finally, the AP is in charge to transmit the data to the cloud
   using the network part.

   The network part is considered as the medium that link the IoT part to the cloud. It is composed
   of several network switches and router and it is considered to be a wired network.

   #+END_COMMENT

      
** Cloud Infrastructure

* System Model [2 col]

  The system model is divided in two parts. First, the IoT and the Network part are models through
  simulations. Then, the Cloud part is model using real servers connected to watt-meters. In this way,
  it is possible to evaluate the end-to-end energy consumption of the system.

** IoT Part
   In the first place, the IoT part is composed of several sensors connected to an Access Point (AP)
   which forms a cell. This cell is model using the ns-3 network simulator. Consequently, we setup
   between 5 and 15 sensors connected to the AP using WIFI 5GHz 802.11n. The node are placed
   randomly in a rectangle of 400m2 around the AP which corresponds to a typical real use case. All
   the cell nodes are setup with the default WIFI energy model provided by ns-3. The different
   energy values used by the energy model are provided on Table \ref{tab:wifi-energy}. These energy
   were extracted from previous work\cite{halperin_demystifying_nodate,li_end--end_2018} on
   802.11n. Besides, we suppose that the energy source of each nodes are unlimited and thus each of
   them can communicate until the end of all the simulations.

   As a scenario, sensors send 192 bits packets to the AP composed of: \textbf{1)} A 128 bits
   sensors id \textbf{2)} A 32 bits integer representing the temperature \textbf{3)} An integer
   timestamp representing the temperature sensing time to store them as time series. The data are
   transmitted immediately at each sensing interval $I$ varied from 1s to 60s. Finally, the AP is in
   charge of relaying data to the cloud via the network part.
   
   #+BEGIN_EXPORT latex
   \begin{table}[]
     \centering
     \caption{Simulations Energy Parameters}
     \label{tab:wifi-energy}
     \subtable[Wifi]{
       \begin{tabular}{@{}lr@{}}
         Parameter      & Value  \\ \midrule
         Supply Voltage & 3.3V   \\
         Tx             & 0.38A  \\
         Rx             & 0.313A \\
         Idle           & 0.273A \\ \bottomrule
     \end{tabular}}
     \hspace{0.3cm}
     \subtable[Network]{
       \label{tab:net-energy}
       \begin{tabular}{@{}lr@{}}
         Parameter  & Value \\ \midrule
         Idle       & 1W           \\ 
         Bytes (Tx/Rx)  & 3.4nJ           \\ 
         Pkt (Tx/Rx)    & 192.0nJ         \\ \bottomrule
       \end{tabular}
     }
   \end{table}
   #+END_EXPORT

** Network Part
   The network part represents the a network section starting from the AP to the Cloud excluding the
   server. It is also model into ns-3. We consider the server to be 9 hops away from the AP with a
   typical round-trip latency of 100ms from the AP to the server. Each node from the AP to the Cloud
   is assume to be network switches with static and dynamic network energy consumption. The first 8
   hop are edge switches and the last one is consider to be a core switch as mention in
   \cite{jalali_fog_2016}. ECOFEN \cite{orgerie_ecofen:_2011} is used to model the energy
   consumption of the network part. ECOFEN is a ns-3 network energy module dedicated to wired
   network. It is based on an energy-per-bit model including static energy consumption by assuming a
   linear relation between the amount of data sent to the network interface and its power
   consumption. The different energy values used to instantiate the ECOFEN energy model for the
   network part are shown in Table \ref{tab:net-energy} and come from previous work
   \cite{cornea_studying_2014-1}.

** Cloud Part
   Finally, to measure the energy consumed by the server, we used real server from the large-scale
   test-beds Grid5000 (G5K). In fact, G5K has a cluster called Nova composed of several nodes which
   are connected to watt-meters. In this way, we can benefit from real energy measurements. The
   server used in the experiment include an Intel Xeon E5-2620 processor with 64 GB of RAM and 600GB
   of disk space on a Linux based operating system. This server is configured to use KVM as
   virtualization mechanism. We deploy a classical Linux x86_64 distribution on the Virtual Machine
   (VM) along with a MySQL database. We used different amount of allocated memory for the VM namely
   1024MB/2048MB/4096MB to highlight its effects on the server energy consumption.

   The sensors requests are simulated using another server. This server is in charge to send hundred
   of requests to the VM in order to fill the database. Consequently, it is easy to vary the
   different requests characteristics namely: \textbf{1)} The number request, to virtually
   add/remove sensors \textbf{2)} The requests interval.

* Evaluation [3 col]
** IoT/Network Consumption
   In a first place, we start by studying the impact of the sensors position on their energy
   consumption. To this end, we run several simulations in ns-3 with different sensors position. The
   results provided by Table \ref{tab:sensorsSendIntervalEffects} show that sensors position have a very low impact
   on the energy consumption and on the application delay. It has an impact of course, but it is very
   limited. This due to the fact that in such a scenario with very small number of communications
   spread over the time, sensors don't have to contend for accessing to the Wifi channel.


   #+BEGIN_EXPORT latex
   % Please add the following required packages to your document preamble:
   % \usepackage{booktabs}
   \begin{table*}[]
   \centering
   \caption{Sensors send interval effects}
   \label{tab:sensorsSendIntervalEffects}
   \begin{tabular}{@{}lrrrrr@{}}
   \toprule
   Sensors Send Interval      & 10s      & 30s      & 50s      & 70s      & 90s      \\ \midrule
   Sensors Power Consumption  & 13.517\hl{94}W & 13.517\hl{67}W & 13.51767W & 13.51767W & 13.517\hl{61}W \\
   Network Power Consumption  & 10.441\hl{78}W & 10.441\hl{67}W & 10.44161W & 10.44161W & 10.441\hl{61}W \\
   Average Appplication Delay & 17.81360s & 5.91265s  & 3.53509s  & 2.55086s  & 1.93848s  \\ \bottomrule
   \end{tabular}
   \end{table*}
   #+END_EXPORT


   Previous work \cite{li_end--end_2018} on similar scenario shows that increasing application
   accuracy impact strongly the energy consumption in the context of data stream analysis. However,
   in our case, application accuracy is driven by the sensing interval and thus, the transmit
   frequency of the sensors. Therefore, we varied the transmission interval of the sensors from 1s
   to 60s. Some of these results are proposed on Table \ref{tab:sensorsSendIntervalEffects}.  In
   case of small and sporadic network traffic, these results show that with a reasonable
   transmission interval the energy consumption of the IoT/Network if almost not affected by the
   variation of this transmission interval. In fact, transmitted data are not large enough to
   leverage the energy consumed by the network.

  

   The number of sensors is a dominant factor that leverage the energy consumption of the
   IoT/Network part. Therefore, we varied the number of sensors in the Wifi cell to analyze its
   impact. The Figure \ref{fig:sensorsNumber} represents the energy consumed by each simulated part
   according the the number of sensors. It is clear that the energy consumed by the network is the
   dominant part. However, since the number of sensors is increasing the energy consumed by the
   network will become negligible face to the energy consume by the sensors. In fact, deploying new
   sensors in the cell do not introduce much network load. To this end, sensors energy consumption
   is dominant.
   
    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[width=0.6\linewidth]{./plots/numberSensors-WIFINET.png}
      \caption{Analysis of the variation of the number of sensors on the IoT/Network part energy consumption.}
      \label{fig:sensorsNumber}
    \end{figure}
    #+END_EXPORT


** Cloud Energy Consumption
   In this End-To-End energy consumption study, cloud account for a huge part of the overall energy
   consumption. According a report \cite{shehabi_united_2016-1} on United States data center energy
   usage, the average Power Usage Effectiveness (PUE) of an hyper-scale data center is 1.2. Thus, in
   our analysis, all energy measurement on cloud server will account for this PUE.

   In a first place, we analyze the impact of the VM allocated memory on the server energy
   consumption. Figure \ref{fig:vmSize} depict the server energy consumption according to the VM
   allocated memory for 20 sensors sending data every 10s. Note that horizontal red line represent
   the average energy consumption for the considered sample of energy values. We can see that at
   each sensing interval, server face to peaks of energy consumption. However, VM allocated memory
   do not influence energy consumption. In fact, simple database requests do not need any particular
   huge memory access and processing time. Thus, remaining experiments are based on VM with 1024MB
   of allocated memory.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \includegraphics[width=0.8\linewidth]{./plots/vmSize-cloud.png}
     \caption{VM size impact on the server energy consumption using 20 sensors sending data every 10s}
     \label{fig:vmSize}
   \end{figure}
   #+END_EXPORT

   Next, we study the effects of increasing the number of sensors on the server energy consumption.
   Figure \ref{fig:sensorsNumber-server} present the results of the average server energy
   consumption when varying the number of sensors from 20 to 500 while Figure
   \ref{fig:sensorsNumber-WPS} present the average server energy cost per sensors according to the
   number of sensors. These results show a clear linear relation between the number of sensors and
   the server energy consumption. Moreover, we can see that the more sensors we have per server, the
   more energy we can save. In fact, since the idle server energy consumption is high, it is more
   energy efficient to maximize the number of sensors per server. As shown on Figure
   \ref{fig:sensorsNumber-WPS}, a significant amount of energy can be save when passing from 20 to
   300 sensors per server.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \subfigure[Average server energy consumption]{
       \includegraphics[width=0.4\linewidth]{./plots/sensorsNumberLine-cloud.png}
       \label{fig:sensorsNumber-server}
     }
     \hspace{0.5cm}
     \subfigure[Average sensors energy cost on server]{
       \includegraphics[width=0.4\linewidth]{./plots/WPS-cloud.png}
       \label{fig:sensorsNumber-WPS}
     }
     \caption{Server energy consumption for sensors sending data every 10s}
     \label{fig:sensorsNumber-cloud}
   \end{figure}
   #+END_EXPORT

   A last parameter can leverage server energy consumption namely sensors send interval. In addition
   to increasing the application accuracy, sensors send interval increase network traffic and
   database accesses. Figure \ref{fig:sensorsFrequency} present the impact on the server energy
   consumption of changing the send interval of 50 sensors to 1s, 10s and 30s. We can see that, the
   lower sensors send interval is, the more server energy consumption peaks occurs. Therefore, it
   leads to an increase of the server energy consumption.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \includegraphics[scale=0.5]{plots/sendInterval-cloud.png}
     \label{fig:sensorsFrequency}
     \caption{Server energy consumption for 50 sensors sending request at different interval.}
   \end{figure}
   #+END_EXPORT
   
** End-To-End Consumption
   
   To have an overview of the energy consume by the system, it is important to consider the
   end-to-end energy consumption. The Figure \ref{fig:end-to-end} represents the end-to-end system
   energy consumption while varying the number of sensors. It is important to see that, for
   small-scale systems, the server energy consumption is dominant face to the energy consumed by the
   sensors. However, since we are using a single server, large-scale sensors deployment lead to an
   increasing consumption of energy in the IoT part. On the other side, network energy consumption
   is stable regarding to the number of sensors since the system use case do not required large data
   transfer. Thus, it is important to remember that, to save energy, we should maximize the number
   of sensors handle by each cloud server while keeping reasonable sensors request intervals.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \hspace{1cm}
     \includegraphics[scale=0.3]{plots/final.png}
     \label{fig:end-to-end}
     \caption{End-to-end network energy consumption using sensors interval of 10s}
   \end{figure}
   #+END_EXPORT
   


* Discussion [1 col]
* Conclusion [1 col]
* References [1 col]
\bibliographystyle{IEEEtran}
\bibliography{references}


* Data Provenance :noexport:
  :PROPERTIES:
  :header-args: :eval never-export
  :END:
** Data Analysis (R Scripts)
*** Utils
**** R
      RUtils is intended to load logs (data.csv) and providing
      simple plot function for them.

      #+NAME: RUtils
      #+BEGIN_SRC R  :eval never
        library("tidyverse")

        # Fell free to update the following
        labels=c(time="Time (s)",sensorsSendInterval="Sensors Send Interval (s)", sensorsNumber="Number of sensors")
        PUE=1.2
        ns3SimTime=1800
        g5kSimTime=300

        loadData=function(path){
          data=read_csv(path)
          if("sensorsEnergy"%in%colnames(data)){ # If it is ns3 logs
            data=data%>%mutate(sensorsEnergy=sensorsEnergy/ns3SimTime)  # Convert to watts
            data=data%>%mutate(networkEnergy=networkEnergy/ns3SimTime)
            data=data%>%mutate(networkEnergy=networkEnergy+getSwitchesIDLE(sensorsNumber,sensorsSendInterval)) # Add Idle conso of switches
            data=data%>%mutate(totalEnergy=totalEnergy/ns3SimTime)
          }
          else{ # Log from g5k
            data=data%>%mutate(energy=energy*PUE) # Take into account PUE
            data=data%>%filter(time<=g5kSimTime) # Remove extrats values (theorical sim time != real sim time)
          }
        }


        getSwitchesIDLE=function(nbSensors, sendInterval){
          pktSize=192
          nEdgeRouter=8
          nCoreRouter=1

          EdgeIdle=4095
          EdgeMax=4550
          EdgeTraffic=560*10^9

          CoreIdle=11070
          CoreMax=12300
          CoreTraffic=4480*10^9

          # Apply 0.6 factor
          EdgeTraffic=EdgeTraffic*0.6
          CoreTraffic=CoreTraffic*0.6

          totalTraffic=pktSize/sendInterval*nbSensors

          EdgeConso=EdgeIdle*(totalTraffic/EdgeTraffic)
          CoreConso=CoreIdle*(totalTraffic/CoreTraffic)

          return(EdgeConso+CoreConso)
        }

                # Get label according to varName
        getLabel=function(varName){
          if(is.na(labels[varName])){
            return(varName)
          }
          return(labels[varName])
        }

        applyTheme=function(plot,...){
          palette<- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")
          plot=plot+
            theme_bw(...)+
              scale_fill_manual(values=palette)+
                scale_colour_manual(values=palette)
          return(plot)
        }

      #+END_SRC



**** Bash
***** Plots -> PDF
     Merge all plots in plots/ folder into a pdf file.
     #+NAME: plotToPDF
     #+BEGIN_SRC bash :results output :noweb yes
       orgFile="plots/plots.org"
       <<singleRun>> # To get all default arguments

       # Write helper function
       function write {
           echo "$1" >> $orgFile
       }

       echo "#+TITLE: Analysis" > $orgFile
       write "#+LATEX_HEADER: \usepackage{fullpage}"
       write "#+OPTIONS: toc:nil"
       # Default arguments
       write '\begin{center}'
       write '\begin{tabular}{lr}'
       write 'Parameters & Values\\'
       write '\hline'
       write "sensorsPktSize & ${sensorsPktSize} bytes\\\\"
       write "sensorsSendInterval & ${sensorsSendInterval}s\\\\"
       write "sensorsNumber & ${sensorsNumber}\\\\"
       write "nbHop & ${nbHop}\\\\"
       write "linksBandwidth & ${linksBandwidth}Mbps\\\\"
       write "linksLatency & ${linksLatency}ms\\\\"
       write '\end{tabular}'
       write '\newline'
       write '\end{center}'

       for plot in $(find plots/ -type f -name "*.png")
       do
           write "\includegraphics[width=0.5\linewidth]{$(basename ${plot})}"
       done

       # Export to pdf
       emacs $orgFile --batch -f org-latex-export-to-pdf --kill  
     #+END_SRC

***** CSVs -> data.csv (G5K)
     Merge all energy file into one (and add additional fields).

     #+NAME: G5K-mergeCSV
     #+BEGIN_SRC sh
       #!/bin/bash

       whichLog="last"

       logsLocation="logs/g5k"
       whichLog="${logsLocation}/${whichLog}"



       logFile="$(dirname $(readlink -f $0))"/$whichLog/simLogs.txt
       dataFile=$(dirname "$logFile")/data.csv


       getValue () {
           line=$(echo "$1" | grep "Simulation para"|sed "s/Simulation parameters: //g")
           key=$2
           echo "$line"|awk 'BEGIN{RS=" ";FS=":"}"'$key'"==$1{gsub("\n","",$0);print $2}'
       }

       ##### Add extract info to energy #####
       IFS=$'\n'
       for cmd in $(cat $logFile|grep "Simulation parameters")
       do
           nodeName=$(getValue $cmd serverNodeName)
           from=$(getValue $cmd simStart)
           to=$(getValue $cmd simEnd)
           vmSize=$(getValue $cmd vmSize)
           nbSensors=$(getValue $cmd nbSensors)
           simKey=$(getValue $cmd simKey)
           sendInterval=$(getValue $cmd sensorsSendInterval)
           csvFile="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}.csv"
           csvFileIDLE="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}_IDLE.csv"
           tmpFile=${csvFile}_tmp
           echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $tmpFile
           minTs=$(tail -n+2 $csvFile|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           minTsIDLE=$(tail -n+2 $csvFileIDLE|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           tail -n+2 ${csvFile} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTs'",sim,"'$sendInterval'}' >> $tmpFile
           tail -n+2 ${csvFileIDLE} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTsIDLE'",IDLE,"'$sendInterval'}' >> $tmpFile
       done


       ##### Fill dataFile #####
       echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $dataFile
       for tmpFile in $(find ${whichLog}/*_tmp -type f)
       do
           tail -n+2 $tmpFile >> $dataFile
           rm $tmpFile # Pay attention to this line :D
       done
     #+END_SRC

     #+RESULTS: G5K-mergeCSV

     #+RESULTS: mergeCSV



***** Log -> data.csv (ns3)
      logToCSV extract usefull data from logs and put them into logs/data.csv.

      #+NAME: NS3-logToCSV
      #+BEGIN_SRC bash :results output
        logsFolder="./logs/ns3/last/"
        csvOutput="$logsFolder/data.csv"

        # First save csv header line
        aLog=$(find $logsFolder -type f -name "*.org"|head -n 1)
        metrics=$(cat $aLog|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
        echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[1]);if(i<NF)printf(",");else{print("")}}}' > $csvOutput

        # Second save all values
        for logFile in $(find $logsFolder -type f -name "*.org")
        do
            metrics=$(cat $logFile|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
            echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[2]);if(i<NF)printf(",");else{print("")}}}' >> $csvOutput
        done
      #+END_SRC

      #+RESULTS: NS3-logToCSV



   
   
*** Plot Scripts
**** Random R Scripts

    Table sensorsSendInterval~Sensors+NetEnergyconsumption
    #+BEGIN_SRC R :noweb yes :results output
      <<RUtils>>

      data=loadData("logs/ns3/last/data.csv")
      sensorsE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,sensorsEnergy)%>%arrange(sensorsSendInterval)
      delay=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,avgDelay)%>%arrange(sensorsSendInterval)
      netE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,networkEnergy)%>%arrange(sensorsSendInterval)
      formatData=right_join(sensorsE,netE)%>%right_join(delay)%>%filter(((sensorsSendInterval/10)%%2)!=0)

      print(t(formatData))

    #+END_SRC

    #+RESULTS:
    :                         [,1]     [,2]     [,3]     [,4]     [,5]
    : sensorsSendInterval 10.00000 30.00000 50.00000 70.00000 90.00000
    : sensorsEnergy       13.51794 13.51767 13.51767 13.51767 13.51761
    : networkEnergy       10.44178 10.44167 10.44161 10.44161 10.44161
    : avgDelay            17.81360  5.91265  3.53509  2.55086  1.93848




    Figure Sensors Position ~ Energy/Delay
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsPosition-delayenergy.png
      <<RUtils>>
      tr=11 # Offset to center delay plot
      data=loadData("logs/ns3/last/data.csv")

      data=data%>%filter(simKey=="SENSORSPOS",sensorsNumber==9)

      p=ggplot(data,aes(y=sensorsEnergy,x=positionSeed,color="Energy"))+xlab(getLabel("Sensors Position Seed"))+ylab(getLabel("Sensors Power Consumption (W)"))+
      geom_line()+geom_point()+geom_line(aes(y=(avgDelay-tr),color="Delay"))+geom_point(aes(y=(avgDelay-tr),color="Delay"))+expand_limits(y=c(0,15))+
      scale_y_continuous(sec.axis = sec_axis(~.+tr, name = "Application Delay (s)")) +
      guides(color=guide_legend(title="Curves"))

      p=applyTheme(p)      
      p=p+theme(axis.title.y.right = element_text(margin = margin(t = 0, r = -8, b = 0, l = 10)))



      ggsave("plots/sensorsPosition-delayenergy.png",dpi=80, width=4, height=3.2)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsPosition-delayenergy.png]]



       Watt per sensor on server
    #+BEGIN_SRC R :noweb yes :results output
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


      data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
      data=data%>%distinct(nbSensors,.keep_all=TRUE)
      data=data%>%mutate(WPS=(avgEnergy/nbSensors))

      print(data%>%select(WPS,nbSensors))
    #+END_SRC



     Impact of vm size
     #+BEGIN_SRC R :results  graphics :file plots/vmSizeBar-cloud.png
       library("tidyverse")
       PUE=1.2
       # Load data
       data=read_csv("./logs/g5k/last/data.csv")

       data=data%>%filter(state=="sim",simKey=="vmSize")%>%mutate(energy=PUE*energy)%>%filter(time<=300)

       data=data%>%group_by(vmSize)%>%mutate(energy=mean(energy))%>%slice(1L)%>%ungroup()
       data=data%>%mutate(vmSize=as.character(vmSize))

       ggplot(data) + geom_bar(aes(x=vmSize,y=energy),stat="identity")+expand_limits(y=c(75,100))+ylab("Server Power Consumption (W)")+
       xlab("Experiment Time (s)")+scale_y_log10()


       ggsave("plots/vmSizeBar-cloud.png",dpi=90,height=3,width=6)  
     #+END_SRC

     #+RESULTS:
     [[file:plots/vmSizeBar-cloud.png]]



     #+NAME: ssiNet
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-net.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=networkEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("networkEnergy"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-net.png",dpi=80)
     #+END_SRC


     Effect of the number of sensors on the application delay
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/delay-nbsensors.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="NBSENSORS") %>% ggplot(aes(y=avgDelay,x=sensorsNumber))+xlab(getLabel("sensorsNumber"))+ylab(getLabel("avgDelay"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/delay-nbsensors.png",dpi=80)
     #+END_SRC

     #+RESULTS:
     [[file:plots/delay-nbsensors.png]]


   
     #+NAME: ssiWifi
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-wifi.png
       <<NS3-RUtils>>
       data=read_csv("logs/ns3/last/data.csv")
       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=sensorsEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("sensorsEnergy"))+
       geom_line() +     geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-wifi.png",dpi=80)
     #+END_SRC


      #+BEGIN_SRC R :results graphics  :noweb yes  :file plots/plot-final.png :session *R*
        <<NS3-RUtils>>
        simTime=1800

        # Network
        data=read_csv("logs/ns3/last/data.csv")
        data=data%>%filter(simKey=="NBSENSORS")
        dataC5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataC10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataNet=rbind(dataC5,dataC10)%>%mutate(type="Network")

        # Network
        dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS=rbind(dataS5,dataS10)%>%mutate(type="Sensors")
      
        data=rbind(dataNet,dataS)%>%mutate(sensorsNumber=as.character(sensorsNumber))

        ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="Part"))
        ggsave("plots/plot-final.png",dpi=80)

      #+END_SRC
    
**** Plot In Paper

     Power sensors vs network
     #+BEGIN_SRC R :noweb yes :results graphics  :file plots/numberSensors-WIFINET.png :session *R*
       <<RUtils>>

       data=loadData("logs/ns3/last/data.csv")
       data=data%>%filter(simKey=="NBSENSORS")
       dataW=data%>%mutate(energy=sensorsEnergy)%>% mutate(type="Sensors") %>% select(sensorsNumber,energy,type) 
       dataN=data%>%mutate(energy=networkEnergy)%>% mutate(type="Network") %>% select(sensorsNumber,energy,type)

       data=rbind(dataN,dataW)
       data=data%>%mutate(sensorsNumber=as.character(sensorsNumber))
       data=data%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
       data=data%>%filter(sensorsNumber%in%c(2,4,6,8,10))

       p=ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),position="dodge",stat="identity")+
       xlab(getLabel("sensorsNumber"))+ ylab("Power Consumption (W)") + guides(fill=guide_legend(title=""))
       p=applyTheme(p)+theme(text = element_text(size=15))

       size=5
      ggsave("plots/numberSensors-WIFINET.png",dpi=90,width=size,height=size-1)
     #+END_SRC

     #+RESULTS:
     [[file:plots/numberSensors-WIFINET.png]]


    Final plot: Energy cloud, network and sensors
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/final.png
      <<RUtils>>

      # Linear Approx
      approx=function(data1, data2,nbSensors){
        x1=data1$sensorsNumber
        y1=data1$energy

        x2=data2$sensorsNumber
        y2=data2$energy

        a=((y2-y1)/(x2-x1))
        b=y1-a*x1

        return(a*nbSensors+b)

      }


      # Load data
      data=loadData("./logs/g5k/last/data.csv")
      data=data%>%filter(state=="sim",simKey=="nbSensors")

      # Cloud
      data20=data%>%filter(nbSensors==20)%>%mutate(energy=mean(energy)) %>% slice(1L)
      data100=data%>%filter(nbSensors==100)%>%mutate(energy=mean(energy)) %>% slice(1L)
      data300=data%>%filter(nbSensors==300)%>%mutate(energy=mean(energy)) %>% slice(1L)
      dataCloud=rbind(data20,data100,data300)%>%mutate(sensorsNumber=nbSensors)%>%mutate(type="Cloud")%>%select(sensorsNumber,energy,type)


      # Network
      data=loadData("./logs/ns3/last/data.csv")
      data=data%>%filter(simKey=="NBSENSORS")
      dataN5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
      dataN10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
      dataNet=rbind(dataN5,dataN10)
      fakeNet=tibble(sensorsNumber=c(20,100,300))
      fakeNet=fakeNet%>%mutate(energy=approx(dataN5,dataN10,sensorsNumber),type="Network")

      # Sensors
      dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber) 
      dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber) 
      dataS=rbind(dataS5,dataS10)
      fakeS=tibble(sensorsNumber=c(20,100,300))
      fakeS=fakeNet%>%mutate(energy=approx(dataS5,dataS10,sensorsNumber),type="Sensors")

      # Combine Net/Sensors/Cloud and order factors
      fakeData=rbind(fakeNet,fakeS,dataCloud)
      fakeData=fakeData%>%mutate(sensorsNumber=as.character(sensorsNumber))
      fakeData=fakeData%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
      fakeData$type=factor(fakeData$type,ordered=TRUE,levels=c("Sensors","Network","Cloud"))

      # Plot
      p=ggplot(fakeData)+geom_bar(position="dodge2",colour="black",aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+ 
      xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="System Part"))
      p=applyTheme(p)+theme(text = element_text(size=16))
      ggsave("plots/final.png",dpi=90,width=8,height=5.5)

    #+END_SRC

    #+RESULTS:
    [[file:plots/final.png]]


    Impact of vm size
    #+BEGIN_SRC R :noweb yes :results graphics :noweb yes :file plots/vmSize-cloud.png
      <<RUtils>>

      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="vmSize")%>%filter(time<=300)

      data=data%>%mutate(vmSize=paste0(vmSize," MB"))
      data=data%>%group_by(vmSize)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~vmSize)+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
      xlab("Experiment Time (s)")

      p=applyTheme(p)

      ggsave("plots/vmSize-cloud.png",dpi=90,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/vmSize-cloud.png]]


    Impact of sensors number
    #+BEGIN_SRC R :noweb yes :results  graphics :file plots/sensorsNumber-cloud.png
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()

      data=data%>%mutate(nbSensorsSort=nbSensors)
      data=data%>%mutate(nbSensors=paste0(nbSensors," Sensors"))
      data$nbSensors=fct_reorder(data$nbSensors, data$nbSensorsSort)

      data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~nbSensors)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
      xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)

      p=applyTheme(p)
      ggsave("plots/sensorsNumber-cloud.png",dpi=90,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsNumber-cloud.png]]
  
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsNumberLine-cloud.png :session *R:2*
          <<RUtils>>
          # Load data
          data=loadData("./logs/g5k/last/data.csv")

          data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


          data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
          data=data%>%distinct(nbSensors,.keep_all=TRUE)
          data=data%>%mutate(WPS=(avgEnergy/nbSensors))

          p=ggplot(data,aes(x=nbSensors, y=avgEnergy)) + geom_point() +geom_line()+
          xlab(getLabel("sensorsNumber"))+ylab("Average server power consumption (W)")

          p=applyTheme(p)+theme(text = element_text(size=14))+ expand_limits(y=108)
          ggsave("plots/sensorsNumberLine-cloud.png",dpi=90,height=4.5,width=4)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsNumberLine-cloud.png]]
  
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/WPS-cloud.png :session *R:2*
            <<RUtils>>
            # Load data
            data=loadData("./logs/g5k/last/data.csv")

            data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


            data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
            data=data%>%distinct(nbSensors,.keep_all=TRUE)
            data=data%>%mutate(WPS=(avgEnergy/nbSensors))

            oldNb=data$nbSensors
            data=data%>%mutate(nbSensors=as.character(nbSensors))
            data$nbSensors=fct_reorder(data$nbSensors,oldNb)
            p=ggplot(data,aes(x=nbSensors, y=WPS)) + geom_bar(stat="identity")+
            xlab(getLabel("sensorsNumber"))+ylab("Server power cost per sensors (W)")

            p=applyTheme(p)+theme(text = element_text(size=14))+        theme(axis.title.y = element_text(margin = margin(t = 0, r = 8, b = 0, l = 0)))
            ggsave("plots/WPS-cloud.png",dpi=90,height=4,width=4)
    #+END_SRC

    #+RESULTS:
    [[file:plots/WPS-cloud.png]]

    #+BEGIN_SRC R :noweb yes :results  graphics :file plots/sendInterval-cloud.png
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="sendInterval")%>%ungroup()

      oldSendInterval=data$sendInterval
      data=data%>%mutate(sendInterval=paste0(sendInterval,"s"))
      data$sendInterval=fct_reorder(data$sendInterval,oldSendInterval)

      data=data%>%group_by(sendInterval)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
          print(data)
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~sendInterval)+expand_limits(y=c(0,40))+ylab("Server power consumption (W)")+
      xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)

      p=applyTheme(p)
      ggsave("plots/sendInterval-cloud.png",dpi=120,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/sendInterval-cloud.png]]

  
* Emacs settings :noexport:
  # Local Variables:
  # eval:    (unless (boundp 'org-latex-classes) (setq org-latex-classes nil))
  # eval:    (add-to-list 'org-latex-classes
  #                       '("IEEEtran" "\\documentclass[conference]{IEEEtran}\n \[NO-DEFAULT-PACKAGES]\n \[EXTRA]\n"  ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}")                       ("\\subsubsection{%s}" . "\\subsubsection*{%s}")                       ("\\paragraph{%s}" . "\\paragraph*{%s}")                       ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
  # End: