Probability Distribution Functions
------------------------------------

.. _PDF:

Probability Density Function
=============================

The Probability Density Function (PDF) is a function defined for a random variable :math:`X` such that:

.. math::
   \forall (a,b) \in \mathbb{R}^2,\, P(a \le X \le b) = \int_a^b f_X(x)dx

Properties:

#. :math:`\int_{-\infty}^{+\infty} f_X(x)dx=1`
#. :math:`P(X=a)=\int_{a}^{a} f_X(x)dx=0`

From *property 2*, it can be derived that (`source <http://yallouz.arie.free.fr/terminale_cours/probascont/prob-continue.php>`__):

.. math::
   P(a \le X \le b) &= P(a < X \le b) =P(a \le X < b) =P(a < X < b)

   P(a \ge X) &= P(a > X) = 1-P(a < X) = 1-P(a \le X)

The PDF of a random variable is intimately related to its :ref:`CDF <CDF>` with the following relation:

.. math::
   F_X(x)=\int_{-\infty}^x f_X(t)dt

To illustrate this property let's take an example with the exponential distribution defined as follow:

.. math::
    f(x;\lambda) = \begin{cases}\lambda  e^{ - \lambda x} & x \ge 0,\\ 0 & x < 0.\end{cases}
    
Let's compute its CDF:

.. math::
   F(x;\lambda)=\int_{0}^x \lambda e^{-\lambda t}dt = -\int_{0}^x -\lambda e^{-\lambda t}dt &= - \left[ e^{-\lambda t} \right]_{0}^x

   &=1 -e^{-\lambda x}
   
.. _CDF:

Cumulative Distribution Function
=================================

The Cumulative Distribution Function (CDF) of a random variable :math:`X` is a function :math:`F_X(x)` such that:

.. math::
   F_X(x) &= P(X \leq x)
   
   \lim_{x \to -\infty} F_X(x) &= 0

   \lim_{x \to +\infty} F_X(x) &= 1
   
CDF can be used for:

.. math::
   P(X \in ]a,b]) = P(a < X \leq b) = F_X(b) - F_X(a)

.. note::
   In the definition above, the "less than or equal to" sign, ":math:`\le`", is a convention.
   Since :math:`F_X(x)` is continuous on :math:`[0,1]` it is similar to :ref:`what is cover for the PDF <PDF>`.
   It is not a universal convention but the distinction is important for discrete distributions.
   More infos `here <https://en.wikipedia.org/wiki/Cumulative_distribution_function>`__.