paper-lowrate-iot/2019-Mascots.org

#+TITLE: Estimating the end-to-end energy consumption of IoT devices along with their impact on Cloud and telecommunication infrastructures

#+EXPORT_EXCLUDE_TAGS: noexport
#+STARTUP: hideblocks
#+OPTIONS: H:5 author:nil email:nil creator:nil timestamp:nil skip:nil toc:nil ^:nil
#+LATEX_CLASS: IEEEtran
#+LATEX_HEADER: \usepackage{hyperref}
#+LATEX_HEADER: \usepackage{booktabs}
#+LATEX_HEADER: \usepackage{subfigure}
#+LATEX_HEADER: \usepackage{graphicx}
#+LATEX_HEADER: \IEEEoverridecommandlockouts 
#+LATEX_HEADER: \author{\IEEEauthorblockN{1\textsuperscript{st} Anne-Cécile Orgerie}
#+LATEX_HEADER: \IEEEauthorblockA{\textit{Univ Rennes, Inria, CNRS, IRISA, Rennes, France} \\
#+LATEX_HEADER: Rennes, France \\
#+LATEX_HEADER: anne-cecile.orgerie@irisa.fr}
#+LATEX_HEADER: \and
#+LATEX_HEADER: \IEEEauthorblockN{2\textsuperscript{nd} Loic Guegan}
#+LATEX_HEADER: \IEEEauthorblockA{\textit{Univ Rennes, Inria, CNRS, IRISA, Rennes, France} \\
#+LATEX_HEADER: Rennes, France \\
#+LATEX_HEADER: loic.guegan@irisa.fr}
#+LATEX_HEADER: }



#+BEGIN_EXPORT latex
\begin{abstract}
Information and Communication Technology takes a growing part in the worldwide energy consumption. One of the root causes of this increase lies in the multiplication of connected devices. Each object of the Internet-of-Things often does not consume much energy by itself. Yet, their number and the infrastructures they require to properly work have leverage. In this paper, we combine simulations and real measurements to study the energy impact of IoT devices. In particular, we analyze the energy consumption of Cloud and telecommunication infrastructures induced by the utilization of connected devices, and we propose an end-to-end energy consumption model for these devices. 
\end{abstract}

\begin{IEEEkeywords}
component, formatting, style, styling, insert
\end{IEEEkeywords}
#+END_EXPORT


* Introduction [2 col]
* Related Work [1 col]
* Use-Case [1 col]
** Application Characteristic

   #+BEGIN_COMMENT
   The IoT part is composed of an Access Point (AP), connected to several sensors using WIFI. In the
   system, the IoT part is considered as the part where the system data are created. In fact, the
   data life cycle start when the sensors records their respective local temperature at a frequency
   $f$ and the local timestamp. Then, these data are transmitted through the network along with an
   arbitrary sensor id of 128 bits. Finally, the AP is in charge to transmit the data to the cloud
   using the network part.

   The network part is considered as the medium that link the IoT part to the cloud. It is composed
   of several network switches and router and it is considered to be a wired network.

   #+END_COMMENT

      
** Cloud Infrastructure

* System Model [2 col]

  The system model is divided in two parts. First, the IoT and the Network part are models through
  simulations. Then, the Cloud part is model using real servers connected to watt-meters. In this way,
  it is possible to evaluate the end-to-end energy consumption of the system.

** IoT Part
   In the first place, the IoT part is composed of several sensors connected to an AP which forms a
   cell. It is model using the ns-3 network simulator. Thus, we setup between 5 and 15 sensors
   connected to the AP using WIFI 5GHz 802.11n. The node are placed randomly in a rectangle of 400m2
   around the AP which correspond to a typical real use case. All the nodes of the cell are setup
   with the default WIFI energy model provided by ns-3.  The different energy values used by the
   energy model are provided on Table \ref{tab:wifi-energy}. These energy were extracted from
   previous work\cite{halperin_demystifying_nodate,li_end--end_2018} on 802.11n. Note that we
   suppose that the energy source of the cell nodes are unlimited and thus every nodes can
   communicate until the end of all the simulations.

   As a scenario, sensors send to the AP packets of 192 bits that include: \textbf{1)} A 128 bits
   sensors id \textbf{2)} A 32 bits integer representing the temperature \textbf{3)} An integer
   timestamp representing the temperature sensing time. The data are transmitted immediately at each
   sensing interval $I$ varied from 1s to 60s. Finally, the AP is in charge of relaying data to the
   cloud using the network part.
   
   #+BEGIN_EXPORT latex
   \begin{table}[]
     \centering
     \caption{Simulations Energy Parameters}
     \label{tab:wifi-energy}
     \subtable[Wifi]{
       \begin{tabular}{@{}lr@{}}
         Parameter      & Value  \\ \midrule
         Supply Voltage & 3.3V   \\
         Tx             & 0.38A  \\
         Rx             & 0.313A \\
         Idle           & 0.273A \\ \bottomrule
     \end{tabular}}
     \hspace{0.3cm}
     \subtable[Network]{
       \label{tab:net-energy}
       \begin{tabular}{@{}lr@{}}
         Parameter  & Value \\ \midrule
         Idle       & 1J            \\ 
         Bytes (Tx/Rx)  & 3.4nJ           \\ 
         Pkt (Tx/Rx)    & 192.0nJ         \\ \bottomrule
       \end{tabular}
     }
   \end{table}
   #+END_EXPORT

** Network Part
   The network part represents the network starting from the AP to the Cloud excluding the server.
   It is also model into ns-3. We consider the server to be 9 hops away from the AP with a typical
   round-trip latency of 100ms from the AP to the server. Each node from the AP to the Cloud is
   assume to be network switches with static and dynamic network energy consumption. ECOFEN
   \cite{orgerie_ecofen:_2011} is used to model the energy consumption of the network part. ECOFEN
   is a ns-3 network energy module for ns-3 dedicated to wired network energy estimation. It is
   based on an energy-per-bit model including static consumption by assuming a linear relation
   between the amount of data sent to the network interface and the power consumption. The different
   energy values used to instantiate the ECOFEN energy model for the network part are shown in Table
   \ref{tab:net-energy} and come from previous work \cite{cornea_studying_2014-1}.

** Cloud Part
   Finally, to measure the energy consumption of the server, we used real server from the
   large-scale test-beds Grid5000 (G5K). In fact, G5K has a cluster called Nova composed of several
   nodes which are connected to watt-meters. In this way, we can benefit from real energy
   measurements. The server used in the experiment is composed of Intel Xeon E5-2620 processor with
   64 GB of RAM and 600GB of disk space on a Linux based distribution. This server is configured to
   use KVM as virtualization mechanism. We deploy a classical Linux x86_64 distribution on the
   Virtual Machines (VM) along with a MySQL database. We different amount of allocated memory for
   the VM namely 1024MB/2048MB/4096MB to highlight its effects on the server energy consumption.

   The sensors requests are simulated using another server. This server is in charge to send hundred
   of requests to the VM in order to fill the database. Consequently, it is easy to vary the
   different requests characteristics namely: \textbf{1)} The number request, to virtually
   add/remove sensors \textbf{2)} The requests frequency.

* Evaluation [3 col]
** IoT/Network Consumption
   In a first place, we start by studying the impact of the sensors position on their energy
   consumption. To this end, we run several simulations in ns-3 with different sensors position. The
   results provided by Figure \ref{fig:sensorsPos} show that sensors position have a very low impact
   on the energy consumption and on the application delay. It has an impact of course but it is very
   limited. This due to the fact that in such a scenario with very small number of communications
   spread over the time, sensors don't have to contend for accessing to the Wifi channel.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \includegraphics[width=0.6\linewidth]{./plots/sensorsPosition-delayenergy.png}
     \caption{Effects of sensors position on the application delay and the sensors energy consumption in a cell of 9 sensors.}
     \label{fig:sensorsPos}
   \end{figure}
   #+END_EXPORT

    Previous work \cite{li_end--end_2018} on similar scenario shows that increasing application
    accuracy impact strongly the energy consumption in the context of data stream analysis. However,
    in how case, application accuracy is driven by the sensing frequency and thus the transmit
    frequency of the sensors. In this way, we vary the transmission frequency of the sensors from 1s
    to 60s. Figure \ref{fig:frequency} present the effects of the sensors transmission frequency on
    the IoT/Network part energy consumption. In case of small and sporadic network traffic, these
    results show that with a reasonable transmission frequency the energy consumption of the
    IoT/Network if almost not affected by the variation of this frequency.

    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[scale=0.45]{./plots/sendFrequency-energy.png}
      \caption{Sensors send interval and its influence on the IoT/Network part energy consumption.}  
      \label{fig:frequency}
    \end{figure}
    #+END_EXPORT
    

   The number of sensors is the dominant factor that leverage the energy consumption of the
   IoT/Network part. Therefore, we varied the number of sensors in the Wifi cell to analyze its
   impact. The figure \ref{fig:sensorsNumber} represents the energy consumption of each simulated
   part. It is clear that the energy consume by the network is the dominant part. However, since the
   number of sensors is increasing the energy consume by the network will become negligible face to
   the energy consume by the sensors. In fact, deploying new sensors in the cell do not introduce
   much network load. To this end, sensors energy consumption is dominant. 
   
    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[width=0.6\linewidth]{./plots/numberSensors-WIFINET.png}
      \caption{Analysis of the variation of the number of sensors on the IoT/Network part energy consumption.}
      \label{fig:sensorsNumber}
    \end{figure}
    #+END_EXPORT


** Cloud Energy Consumption
   In this End-To-End energy consumption study, cloud account for a huge part of the overall energy
   consumption. According a report \cite{shehabi_united_2016-1} on United States data center energy
   usage, the average Power Usage Effectiveness (PUE) of an hyperscale data center is 1.2. Thus, in
   our analysis, all energy measurement on cloud server will account for this PUE.
   


*** Virtual Machine Size Impact
** End-To-End Consumption
* Discussion [1 col]
* Conclusion [1 col]
* References [1 col]
\bibliographystyle{IEEEtran}
\bibliography{references}


* Data Provenance :noexport:
  :PROPERTIES:
  :header-args: :eval never-export
  :END:
** Data Analysis
*** NS3
    To Generate all the plots, please execute the following line:
    #+NAME: runAnalysis
    #+CALL: plotToPDF(plots=genAllPlots(data=NS3-logToCSV()))

    #+RESULTS: runAnalysis

**** R Scripts
***** Generate all plots script
      Available variables:
      |---------------------|
      | Name                |
      |---------------------|
      | sensorsSendInterval |
      | sensorsPktSize      |
      | sensorsNumber       |
      | nbHop               |
      | linksBandwidth      |
      | linksLatency        |
      | totalEnergy         |
      | nbPacketCloud       |
      | nbNodes             |
      | avgDelay            |
      | simKey              |
      |---------------------|

      #+NAME: genAllPlots
      #+BEGIN_SRC R :noweb yes :results output
        <<NS3-RUtils>>
        data=read_csv("logs/ns3/last/data.csv")
  #      easyPlotGroup("linksLatency","totalEnergy", "LATENCY","sensorsNumber")
  #      easyPlotGroup("linksBandwidth","totalEnergy", "BW","sensorsNumber")
        easyPlot("sensorsNumber","totalEnergy", "NBSENSORS")
        easyPlotGroup("positionSeed", "totalEnergy","SENSORSPOS","sensorsNumber")
        easyPlotGroup("positionSeed", "avgDelay","SENSORSPOS","sensorsNumber")
        easyPlotGroup("sensorsSendInterval","sensorsEnergy","SENDINTERVAL","sensorsNumber")
        easyPlotGroup("sensorsSendInterval","networkEnergy","SENDINTERVAL","sensorsNumber")

      #+END_SRC

      #+RESULTS: genAllPlots

***** R Utils
      RUtils is intended to load logs (data.csv) and providing
      simple plot function for them.

      #+NAME: NS3-RUtils
      #+BEGIN_SRC R :eval never
        library("tidyverse")

        # Fell free to update the following
        labels=c(nbNodes="Number of nodes",sensorsNumber="Number of sensors",totalEnergy="Total Energy (J)",
          nbHop="Number of hop (AP to Cloud)", linksBandwidth="Links Bandwidth (Mbps)", avgDelay="Average Application Delay (s)",
          linksLatency="Links Latency (ms)", sensorsSendInterval="Sensors Send Interval (s)", positionSeed="Position Seed",
          sensorsEnergy="Sensors Wifi Energy Consumption (J)", networkEnergy="Network Energy Consumption (J)")

        # Get label according to varName
        getLabel=function(varName){
          if(is.na(labels[varName])){
            return(varName)
          }
          return(labels[varName])
        }

        easyPlot=function(X,Y,KEY){
          curData=data%>%filter(simKey==KEY)
          stopifnot(NROW(curData)>0)
          ggplot(curData,aes_string(x=X,y=Y))+geom_point()+geom_line()+xlab(getLabel(X))+ylab(getLabel(Y))
          ggsave(paste0("plots/",KEY,"-",X,"_",Y,".png"))
        }

        easyPlotGroup=function(X,Y,KEY,GRP){
          curData=data%>%filter(simKey==KEY) %>% mutate(!!GRP:=as.character(UQ(rlang::sym(GRP)))) # %>%mutate(sensorsNumber=as.character(sensorsNumber))
          stopifnot(NROW(curData)>0)
          ggplot(curData,aes_string(x=X,y=Y,color=GRP,group=GRP))+geom_point()+geom_line()+xlab(getLabel(X))+ylab(getLabel(Y)) + labs(color = getLabel(GRP))
          ggsave(paste0("plots/",KEY,"-",X,"_",Y,".png"))
        }
      #+END_SRC

**** Plots -> PDF
     Merge all plots in plots/ folder into a pdf file.
     #+NAME: plotToPDF
     #+BEGIN_SRC bash :results output :noweb yes
       orgFile="plots/plots.org"
       <<singleRun>> # To get all default arguments

       # Write helper function
       function write {
           echo "$1" >> $orgFile
       }

       echo "#+TITLE: Analysis" > $orgFile
       write "#+LATEX_HEADER: \usepackage{fullpage}"
       write "#+OPTIONS: toc:nil"
       # Default arguments
       write '\begin{center}'
       write '\begin{tabular}{lr}'
       write 'Parameters & Values\\'
       write '\hline'
       write "sensorsPktSize & ${sensorsPktSize} bytes\\\\"
       write "sensorsSendInterval & ${sensorsSendInterval}s\\\\"
       write "sensorsNumber & ${sensorsNumber}\\\\"
       write "nbHop & ${nbHop}\\\\"
       write "linksBandwidth & ${linksBandwidth}Mbps\\\\"
       write "linksLatency & ${linksLatency}ms\\\\"
       write '\end{tabular}'
       write '\newline'
       write '\end{center}'

       for plot in $(find plots/ -type f -name "*.png")
       do
           write "\includegraphics[width=0.5\linewidth]{$(basename ${plot})}"
       done

       # Export to pdf
       emacs $orgFile --batch -f org-latex-export-to-pdf --kill  
     #+END_SRC

     #+RESULTS:

**** Log -> CSV
     logToCSV extract usefull data from logs and put them into logs/data.csv.

     #+NAME: NS3-logToCSV
     #+BEGIN_SRC bash :results output
       logsFolder="./logs/ns3/last/"
       csvOutput="$logsFolder/data.csv"

       # First save csv header line
       aLog=$(find $logsFolder -type f -name "*.org"|head -n 1)
       metrics=$(cat $aLog|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
       echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[1]);if(i<NF)printf(",");else{print("")}}}' > $csvOutput

       # Second save all values
       for logFile in $(find $logsFolder -type f -name "*.org")
       do
           metrics=$(cat $logFile|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
           echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[2]);if(i<NF)printf(",");else{print("")}}}' >> $csvOutput
       done
     #+END_SRC

     #+RESULTS: NS3-logToCSV



   
   
   
**** Custom Plots




     Effect of sensors position on app delay
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsPosition-delayenergy.png
       <<NS3-RUtils>>
       simTime=1800


       cbPalette <- c("#0000B0", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7")

       # Load Data
        data=read_csv("logs/ns3/last/data.csv")
        data=data%>%mutate(sensorsEnergyW=sensorsEnergy/simTime)

       data%>%filter(simKey=="SENSORSPOS",sensorsNumber==10) %>% ggplot(aes(y=sensorsEnergyW,x=positionSeed,color="Energy"))+xlab(getLabel("Sensors Position Seed"))+ylab(getLabel("Sensors Energy Consumption (W)"))+
       geom_line()+geom_point()+geom_line(aes(y=(avgDelay+5),color="Delay"))+geom_point(aes(y=(avgDelay+5),color="Delay"))+expand_limits(y=c(0,15))+scale_y_continuous(sec.axis = sec_axis(~.-5, name = "Application Delay (s)")) +theme_bw() +  scale_fill_manual(values=cbPalette) +   scale_colour_manual(values=cbPalette)+guides(color=guide_legend(title="Curves"))
       ggsave("plots/sensorsPosition-delayenergy.png",dpi=80, width=4, height=3.2)
     #+END_SRC

     #+RESULTS:
     [[file:plots/sensorsPosition-delayenergy.png]]




     #+NAME: ssiNet
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-net.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=networkEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("networkEnergy"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-net.png",dpi=80)
     #+END_SRC


     Effect of the number of sensors on the application delay
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/delay-nbsensors.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="NBSENSORS") %>% ggplot(aes(y=avgDelay,x=sensorsNumber))+xlab(getLabel("sensorsNumber"))+ylab(getLabel("avgDelay"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/delay-nbsensors.png",dpi=80)
     #+END_SRC

     #+RESULTS:
     [[file:plots/delay-nbsensors.png]]


   
     #+NAME: ssiWifi
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-wifi.png
       <<NS3-RUtils>>
       data=read_csv("logs/ns3/last/data.csv")
       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=sensorsEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("sensorsEnergy"))+
       geom_line() +     geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-wifi.png",dpi=80)
     #+END_SRC


      #+BEGIN_SRC R :results graphics  :noweb yes  :file plot-final.png :session *R*
        <<NS3-RUtils>>
        simTime=1800

        # Network
        data=read_csv("logs/ns3/last/data.csv")
        data=data%>%filter(simKey=="NBSENSORS")
        dataC5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataC10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataNet=rbind(dataC5,dataC10)%>%mutate(type="Network")

        # Network
        dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS=rbind(dataS5,dataS10)%>%mutate(type="Sensors")
      
        data=rbind(dataNet,dataS)%>%mutate(sensorsNumber=as.character(sensorsNumber))

        ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="Part"))
        ggsave("plot-final.png",dpi=80)

      #+END_SRC

   
*** Cloud
**** R Scripts
***** Plots script
      #+BEGIN_SRC R :results output  :noweb yes  :file second-final/plot.png
        <<RUtils>>
        dataOrig=loadData("./second-final/data.csv")

        data=dataOrig%>%filter(simKey=="nbSensors")%>%filter(state=="sim",nbSensors==100)
        dataIDLE=dataOrig%>%filter(simKey=="nbSensors")%>%filter(state!="sim",nbSensors==100)
        data=data%>%mutate(meanEnergy=mean(energy))
        dataIDLE=dataIDLE%>%mutate(meanEnergy=mean(energy))
        data=rbind(data,dataIDLE)
        ggplot(data,aes(x=time,y=energy))+geom_point(position="jitter")+xlab(getLabel("time"))+expand_limits(y=0)+facet_wrap(~state)+geom_hline(aes(color=state,yintercept=mean(meanEnergy)))
        ggsave("./second-final/plot.png",dpi=180)
      #+END_SRC

      #+RESULTS:
      #+begin_example
      # A tibble: 3,050 x 8
                  ts energy simKey    vmSize nbSensors   time state meanEnergy
               <dbl>  <dbl> <chr>      <dbl>     <dbl>  <dbl> <chr>      <dbl>
       1 1558429001.   90.2 nbSensors   2048       100 0      IDLE        90.8
       2 1558429001.   89   nbSensors   2048       100 0.0199 IDLE        90.8
       3 1558429001.   89   nbSensors   2048       100 0.0399 IDLE        90.8
       4 1558429001.   90.8 nbSensors   2048       100 0.0599 IDLE        90.8
       5 1558429001.   91   nbSensors   2048       100 0.0799 IDLE        90.8
       6 1558429001.   90.5 nbSensors   2048       100 0.1000 IDLE        90.8
       7 1558429001.   89.9 nbSensors   2048       100 0.120  IDLE        90.8
       8 1558429001.   88.6 nbSensors   2048       100 0.140  IDLE        90.8
       9 1558429001.   88.6 nbSensors   2048       100 0.160  IDLE        90.8
      10 1558429001.   90.5 nbSensors   2048       100 0.180  IDLE        90.8
      # … with 3,040 more rows
      #+end_example



****** Custom Plots

     #+NAME: ssiNet
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-net.png
       <<RUtils>>

       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=networkEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("networkEnergy"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-net.png",dpi=80)
     #+END_SRC

     #+RESULTS:
     [[file:plots/sensorsSendInterval-net.png]]

   
     #+NAME: ssiWifi
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-wifi.png
       <<RUtils>>
       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=sensorsEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("sensorsEnergy"))+
       geom_line() +     geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-wifi.png",dpi=80)
     #+END_SRC

     #+RESULTS: ssiWifi
     [[file:plots/sensorsSendInterval-wifi.png]]

     #+RESULTS:
     [[file:plots/sensorsSendInterval.png]]


***** R Utils
      RUtils is intended to load logs (data.csv) and providing
      simple plot function for them.

      #+NAME: G5K-RUtils
      #+BEGIN_SRC R :eval never
        library("tidyverse")

        # Fell free to update the following
        labels=c(time="Time (s)")
   
        loadData=function(path){
          data=read_csv(path)
        }

        # Get label according to varName
        getLabel=function(varName){
          if(is.na(labels[varName])){
            return(varName)
          }
          return(labels[varName])
        }
      #+END_SRC
    
**** Plots -> PDF
     Merge all plots in plots/ folder into a pdf file.
     #+NAME: plotToPDF
     #+BEGIN_SRC bash :results output :noweb yes
       orgFile="plots/plots.org"
       <<singleRun>> # To get all default arguments

       # Write helper function
       function write {
           echo "$1" >> $orgFile
       }

       echo "#+TITLE: Analysis" > $orgFile
       write "#+LATEX_HEADER: \usepackage{fullpage}"
       write "#+OPTIONS: toc:nil"
       # Default arguments
       write '\begin{center}'
       write '\begin{tabular}{lr}'
       write 'Parameters & Values\\'
       write '\hline'
       write "sensorsPktSize & ${sensorsPktSize} bytes\\\\"
       write "sensorsSendInterval & ${sensorsSendInterval}s\\\\"
       write "sensorsNumber & ${sensorsNumber}\\\\"
       write "nbHop & ${nbHop}\\\\"
       write "linksBandwidth & ${linksBandwidth}Mbps\\\\"
       write "linksLatency & ${linksLatency}ms\\\\"
       write '\end{tabular}'
       write '\newline'
       write '\end{center}'

       for plot in $(find plots/ -type f -name "*.png")
       do
           write "\includegraphics[width=0.5\linewidth]{$(basename ${plot})}"
       done

       # Export to pdf
       emacs $orgFile --batch -f org-latex-export-to-pdf --kill  
     #+END_SRC

**** CSVs -> CSV
     Merge all energy file into one (and add additional fields).

     #+NAME: G5K-mergeCSV
     #+BEGIN_SRC sh
       #!/bin/bash

       whichLog="last"

       logsLocation="logs/g5k"
       whichLog="${logsLocation}/${whichLog}"



       logFile="$(dirname $(readlink -f $0))"/$whichLog/simLogs.txt
       dataFile=$(dirname "$logFile")/data.csv


       getValue () {
           line=$(echo "$1" | grep "Simulation para"|sed "s/Simulation parameters: //g")
           key=$2
           echo "$line"|awk 'BEGIN{RS=" ";FS=":"}"'$key'"==$1{gsub("\n","",$0);print $2}'
       }

       ##### Add extract info to energy #####
       IFS=$'\n'
       for cmd in $(cat $logFile|grep "Simulation parameters")
       do
           nodeName=$(getValue $cmd serverNodeName)
           from=$(getValue $cmd simStart)
           to=$(getValue $cmd simEnd)
           vmSize=$(getValue $cmd vmSize)
           nbSensors=$(getValue $cmd nbSensors)
           simKey=$(getValue $cmd simKey)
           csvFile="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}.csv"
           csvFileIDLE="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}_IDLE.csv"
           tmpFile=${csvFile}_tmp
           echo ts,energy,simKey,vmSize,nbSensors,time,state > $tmpFile
           minTs=$(tail -n+2 $csvFile|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           minTsIDLE=$(tail -n+2 $csvFileIDLE|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           tail -n+2 ${csvFile} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTs'",sim"}' >> $tmpFile
           tail -n+2 ${csvFileIDLE} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTsIDLE'",IDLE"}' >> $tmpFile
       done


       ##### Fill dataFile #####
       echo ts,energy,simKey,vmSize,nbSensors,time,state > $dataFile
       for tmpFile in $(find ${whichLog}/*_tmp -type f)
       do
           tail -n+2 $tmpFile >> $dataFile
           rm $tmpFile # Pay attention to this line :D
       done
     #+END_SRC

     #+RESULTS: G5K-mergeCSV

     #+RESULTS: mergeCSV
*** Final Plots


    Figure 
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sendFrequency-energy.png
      <<NS3-RUtils>>
      simTime=1800
      tr=171


      cbPalette <- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")
      # Load Data
       data=read_csv("logs/ns3/last/data.csv")
       data=data%>%mutate(energy=totalEnergy/simTime)

      data=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) 

      ggplot(data,aes(y=energy,x=sensorsSendInterval))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("Sensors And Network\nEnergy Consumption (W)"))+
      geom_line()+geom_point()+expand_limits(y=c(0,50)) +theme_bw() +  scale_fill_manual(values=cbPalette) +   scale_colour_manual(values=cbPalette)+guides(color=guide_legend(title="Curves"))+
      theme(axis.title.y.right = element_text(margin = margin(t = 0, r = -12, b = 0, l = 7)))
      
      ggsave("plots/sendFrequency-energy.png",dpi=100, width=3, height=2.8)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sendFrequency-energy.png]]



    Figure Sensors Position ~ Energy/Delay
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsPosition-delayenergy.png
      <<NS3-RUtils>>
      simTime=1800
      tr=171


      cbPalette <- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")
      # Load Data
       data=read_csv("logs/ns3/last/data.csv")
       data=data%>%mutate(sensorsEnergyW=sensorsEnergy/simTime)

      data%>%filter(simKey=="SENSORSPOS",sensorsNumber==9) %>% ggplot(aes(y=sensorsEnergyW,x=positionSeed,color="Energy"))+xlab(getLabel("Sensors Position Seed"))+ylab(getLabel("Sensors Energy Consumption (W)"))+
      geom_line()+geom_point()+geom_line(aes(y=(avgDelay-tr),color="Delay"))+geom_point(aes(y=(avgDelay-tr),color="Delay"))+expand_limits(y=c(0,15))+
      scale_y_continuous(sec.axis = sec_axis(~.+tr, name = "Application Delay (s)")) +theme_bw() +  scale_fill_manual(values=cbPalette) +   scale_colour_manual(values=cbPalette)+guides(color=guide_legend(title="Curves"))+
      theme(axis.title.y.right = element_text(margin = margin(t = 0, r = -12, b = 0, l = 7)))
      
      ggsave("plots/sensorsPosition-delayenergy.png",dpi=80, width=4, height=3.2)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsPosition-delayenergy.png]]
    



    #+BEGIN_SRC R :noweb yes :results graphics  :file plots/numberSensors-WIFINET.png :session *R*
      <<NS3-RUtils>>
      simTime=1800


      cbPalette <- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")

      # Load Data
      data=read_csv("logs/ns3/last/data.csv")
      data=data%>%filter(simKey=="NBSENSORS")
      dataW=data%>%mutate(energy=sensorsEnergy/simTime)%>% mutate(type="Sensors") %>% select(sensorsNumber,energy,type) 
      dataN=data%>%mutate(energy=networkEnergy/simTime)%>% mutate(type="Network") %>% select(sensorsNumber,energy,type)

      data=rbind(dataN,dataW)
      data=data%>%mutate(sensorsNumber=as.character(sensorsNumber))
      data=data%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
      data=data%>%filter(sensorsNumber%in%c(2,4,6,8,10))

      ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),position="identity",stat="identity")+
      theme_bw()+
          theme(text = element_text(size=16))+
      scale_fill_manual(values=cbPalette) +   scale_colour_manual(values=cbPalette)+
      xlab(getLabel("sensorsNumber"))+ ylab("Energy Consumption (W)") + guides(fill=guide_legend(title="")) +coord_flip()

      size=5
     ggsave("plots/numberSensors-WIFINET.png",dpi=90,width=size,height=size-1)
    #+END_SRC

    #+RESULTS:
    [[file:plots/numberSensors-WIFINET.png]]






  Final plot: Energy cloud, network and sensors
  #+BEGIN_SRC R :noweb yes :results graphics :file plots/final.png :session *R*
    library("tidyverse")

    # Load data
    data=read_csv("./logs/g5k/last/data.csv")
    data=data%>%filter(state=="sim",simKey=="nbSensors")
    PUE=1.2

    # Cloud
    data10=data%>%filter(nbSensors==20)%>%mutate(energy=mean(energy)) %>% slice(1L)
    data100=data%>%filter(nbSensors==100)%>%mutate(energy=mean(energy)) %>% slice(1L)
    data300=data%>%filter(nbSensors==300)%>%mutate(energy=mean(energy)) %>% slice(1L)
    dataCloud=rbind(data10,data100,data300)%>%mutate(sensorsNumber=nbSensors)%>%mutate(type="Cloud")%>%select(sensorsNumber,energy,type)
    dataCloud=dataCloud%>%mutate(energy=energy*PUE)


    approx=function(data1, data2,nbSensors){
      x1=data1$sensorsNumber
      y1=data1$energy

      x2=data2$sensorsNumber
      y2=data2$energy

      a=((y2-y1)/(x2-x1))
      b=y1-a*x1

      return(a*nbSensors+b)

    }


    simTime=1800

    # Network
    data=read_csv("./logs/ns3/last/data.csv")
    data=data%>%filter(simKey=="NBSENSORS")
    dataC5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
    dataC10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
    dataNet=rbind(dataC5,dataC10)%>%mutate(type="Network")

    # Sensors
    dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
    dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
    dataS=rbind(dataS5,dataS10)%>%mutate(type="Sensors")

    fakeNetS=tibble(
      sensorsNumber=c(20,100,300,20,100,300),
      energy=c(dataC10$energy,approx(dataC5,dataC10,100),approx(dataC5,dataC10,300),dataS10$energy,approx(dataS5,dataS10,100),approx(dataS5,dataS10,300)),
      type=c("Network","Network","Network","Sensors","Sensors","Sensors")
    )

    fakeNetS=fakeNetS%>%mutate(sensorsNumber=as.character(sensorsNumber))
    dataCloud=dataCloud%>%mutate(sensorsNumber=as.character(sensorsNumber))

    data=rbind(fakeNetS,dataCloud)%>%mutate(sensorsNumber=as.character(sensorsNumber))


    data=data%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))

    ggplot(data)+geom_bar(position="dodge2",colour="black",aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+ 
    xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="Part"))
    ggsave("plots/final.png",dpi=80)

  #+END_SRC

  #+RESULTS:
  [[file:plots/final.png]]




  #+BEGIN_SRC R :results  graphics :file plots/vmSize-cloud.png
       library("tidyverse")

       # Load data
       data=read_csv("./logs/g5k/last/data.csv")
       data=data%>%filter(simKey=="nbSensors")

       dataI=data%>%filter(state=="IDLE")%>%group_by(nbSensors)%>%mutate(energy=mean(energy))
       dataS=data%>%filter(state=="sim")%>%group_by(nbSensors)%>%mutate(energy=mean(energy))


       data=rbind(dataI,dataS)

#       data1024=data%>%filter(vmSize==1024)%>%mutate(energy=mean(energy))# %>% slice(1L)
#       data100=data%>%filter(nbSensors==100)%>%mutate(energy=mean(energy)) %>% slice(1L)
 #      data300=data%>%filter(nbSensors==300)%>%mutate(energy=mean(energy)) %>% slice(1L)

       ggplot(data,aes(x=time, y=energy,color=state)) + geom_point()+facet_wrap(~nbSensors)



       ggsave("plots/vmSize-cloud.png",dpi=80)  
  #+END_SRC

  #+RESULTS:
  [[file:plots/vmSize-cloud.png]]



* Emacs settings :noexport:
  # Local Variables:
  # eval:    (unless (boundp 'org-latex-classes) (setq org-latex-classes nil))
  # eval:    (add-to-list 'org-latex-classes
  #                       '("IEEEtran" "\\documentclass[conference]{IEEEtran}\n \[NO-DEFAULT-PACKAGES]\n \[EXTRA]\n"  ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}")                       ("\\subsubsection{%s}" . "\\subsubsection*{%s}")                       ("\\paragraph{%s}" . "\\paragraph*{%s}")                       ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
  # End: