This document introduces random variables, focusing on discrete cases before extending to continuous ones. A random variable is a numerical outcome of an experiment; its probability distribution shows the likelihood of each value. The probability mass function, p(x), maps each value to its probability, summing to one. The text distinguishes between random variables (numeric values) and events (occurrences). Expected value E(X) is calculated by summing each value multiplied by its probability, representing the average outcome over many trials. Higher moments, like E(X²), are also introduced, crucial for calculating variance. Variance, measuring the spread of the distribution, is defined as E[(X - E(X))²] = E(X²) - [E(X)]². Standard deviation is the square root of the variance. The moment generating function, MX(t) = E(e^(Xt)), provides a method for easily calculating moments through differentiation. The document then extends these concepts to continuous random variables, replacing sums with integrals and introducing the probability density function f(x), where the integral of f(x) over the range equals 1. Examples using coin flips and dice rolls illustrate the calculations of expected value, variance, and standard deviation, along with the application of the moment generating function.

Random Variables, Expected Value, Variance and Moments
Kevin Burke University of Limerick, Maths & Stats Dept 1 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Random Variables
In probability theory, a random variable is a numerical quantity whose
value is determined by an experiment.
For example, consider the experiment of flipping two coins.
Now define a random variable X = “the number of heads” whose value
will clearly be 0, 1 or 2 heads:
Outcome HH HT TH TT
Value assigned to X 2 1 1 0
Kevin Burke University of Limerick, Maths & Stats Dept 2 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Distribution of a Random Variable
The probability distribution of X is:
x 0 1 2
Pr(X = x)
1
4
1
2
1
4
This describes how likely each of the values are, i.e., how the
probability gets distributed to each possible value of X.
Note that upper case X denotes the random variable whereas lower
case x represents a specific value.
Pr(X = x) means “the probability that the random variable X attains
the specific value x” where x ∈ {0, 1, 2}, e.g., Pr(X = 0) = 1
4
.
Kevin Burke University of Limerick, Maths & Stats Dept 3 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Probability Mass Function
Pr(X = x) is called the probability mass function - it maps each
value of X to a probability value.
This is often shortened to p(x) - pronounced “p - of - x”.
The probability values of this function must sum to one:
Xp(xi) = 1 .
In the previous example, p(0) = 1
4
, p(1) = 1
2
and p(2) = 1
4
.
⇒ p(0) + p(1) + p(2) = 1.
Kevin Burke University of Limerick, Maths & Stats Dept 4 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Random Variable Vs Event
Previously we encountered events - not the same as random variables.
For the sake of clarity consider:

1. The event A = “two heads showing”.
   An event which either occurs or does not occur following the
   experiment.
   It refers to one specific event; we can calculate Pr(A).


2. The random variable X = “the number of heads”.
   A numeric variable whose value is assigned following the experiment.
   Related to X are three events: X = 0, X = 1 and X = 2; we can
   calculate Pr(X = 0), Pr(X = 1) and Pr(X = 2).
   Note: X = 2 is the event A.
   Kevin Burke University of Limerick, Maths & Stats Dept 5 / 31
   Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
   Example: Flipping Two Coins
   Continuing the example of flipping two coins, we could define another
   random variable Y = “the number of unique faces showing”.
   The possible values for this random variable are 1 (if the faces are the
   same) or 2 (if the faces are different):
   Outcome HH HT TH TT
   Value assigned to Y 1 2 2 1
   From the above we get the probability distribution of Y:
   y 1 2
   Pr(Y = y)
   1
   2
   1
   2
   Kevin Burke University of Limerick, Maths & Stats Dept 6 / 31
   Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
   Expected Value
   Just as we calculated the mean as a measure of centrality for a
   distribution of data, we can calculate the expected value for a
   probability distribution.
   The expected value is:
   E(X) = Xxi p(xi) .
   In words: multiply each possible value of X by its probability value and
   then sum the results.
   This is the value we would expect to get on average if we carried out
   the experiment a large number of times.
   Kevin Burke University of Limerick, Maths & Stats Dept 7 / 31
   Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
   The Second Moment
   We will also need to calculate E(X
   2
   ) which is called the second
   moment (E(X) is the first):
   E(X
   2
   ) = Xx
   2
   i p(xi) .
   Note that E(X
   2
   ) is not directly of interest but is used to calculate the
   variance of X.
   Kevin Burke University of Limerick, Maths & Stats Dept 8 / 31
   Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
   Example: Flipping Two Coins
   The random variable X = “the number of heads” has a probability
   distribution given by:
   x 0 1 2
   Pr(X = x)
   1
   4
   1
   2
   1
   4
   ⇒ E(X) = 
   0 ×
   1
   4
   


1 ×
1
2

+

2 ×
1
4


1
2
+
2
4

4
4
= 1.
On average there will be one head showing.
Kevin Burke University of Limerick, Maths & Stats Dept 9 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Example: Flipping Two Coins
We can also calculate:
⇒ E(X
2
) = 
0
2 ×
1
4

+

1
2 ×
1
2

+

2
2 ×
1
4



0 ×
1
4

+

1 ×
1
2

+

4 ×
1
4


1
2
+
4
4
= 1.5.
This value will be used later to calculate the variance.
Kevin Burke University of Limerick, Maths & Stats Dept 10 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Question 1
We continue with the experiment of flipping two coins.
We had the random variable Y = “the number of unique faces”.
y 1 2
Pr(Y = y)
1
2
1
2
a) Calculate E(Y).
b) Calculate E(Y
2
).
Kevin Burke University of Limerick, Maths & Stats Dept 11 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Expectation of Functions of X
Continuing with the previous example, let’s say we wanted to know:
E(X
3
) = 
0
3 ×
1
4

+

1
3 ×
1
2

+

2
3 ×
1
4


1
2
+
8
4
= 2.5,
or
E(e
X
) = 
e
0 ×
1
4

+

e
1 ×
1
2

+

e
2 ×
1
4


1
4
+
e
2
+
e
2
4
≈ 3.46.
In general, the expected value of a g(X) is given by
E[g(X)] = Xg(xi) p(xi) .
Kevin Burke University of Limerick, Maths & Stats Dept 12 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Linear Property of Expectation
An important special case occurs when g(X) is a linear function:
g(X) = aX + b. In this case
E(aX + b) = a E(X) + b .
Proof
E(aX + b) = X(a xi + b)p(xi)

X[a xi p(xi) + b p(xi)]

Xa xi p(xi) +Xb p(xi)
= a
Xxi p(xi) + b
Xp(xi)
= a E(X) + b
since Pxi p(xi) = E(X) and Pp(xi) = 1 by definition.
Kevin Burke University of Limerick, Maths & Stats Dept 13 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Variance and Standard Deviation
Just as we calculated the variance of a set of data, we can calculate
the variance for a probability distribution.
Recall that variance is the average squared distance from the mean:
⇒ Var(X) = E[(X − E(X) )2
] = X( xi − E(X) )2
p(xi).
The above formula can be simplified to
Var(X) = E(X
2
) − [E(X)]2
.
The standard deviation is then
Sd(X) = p
Var(X) .
(reminder: variance is measured in units-squared and standard deviation is in units)
Kevin Burke University of Limerick, Maths & Stats Dept 14 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Variance Formula
var(X) = E[(X − E(X) )2
] = E(X
2
) − [E(X)]2
.
Proof
E[(X − E(X) )2
] = X( xi − E(X) )2
p(xi)

X( x
2
i − 2xiE(X) + [E(X)]2
) p(xi)

X x
2
i p(xi)
| {z }
=E(X2)
−2E(X)
Xxi p(xi)
| {z }
=E(X)
+[E(X)]2X p(xi)
| {z }
=1
= E(X
2
) − 2E(X)E(X) + [E(X)]2
= E(X
2
) − 2[E(X)]2 + [E(X)]2
= E(X
2
) − [E(X)]2
Kevin Burke University of Limerick, Maths & Stats Dept 15 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Variance of a Linear Function
var(aX + b) = a
2
var(X) .
Proof
var(X) = E[ (X − E(X))2
]
⇒ var(aX + b) = E[ (aX + b − E(aX + b))2
]
= E[ (aX + b − aE(X) − b)
2
]
= E[ (aX − aE(X))2
]
= E[ (a (X − E(X)) )2
]
= E[ a
2
(X − E(X))2
]
= a
2E[ (X − E(X))2
]
= a
2
var(X)
Kevin Burke University of Limerick, Maths & Stats Dept 16 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Example: Flipping Two Coins
We continue the example of flipping two coins where X = “the number
of heads”.
We have calculated E(X) = 1 and E(X
2
) = 1.5.
⇒ Var(X) = E(X
2
) − [E(X)]2 = 1.5 − (1)
2 = 1.5 − 1 = 0.5 heads2
,
and the standard deviation is
Sd(X) = p
Var(X) = √
0.5 ≈ 0.707 heads.
Kevin Burke University of Limerick, Maths & Stats Dept 17 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Question 2
We had the random variable Y = “the number of unique faces” based
on flipping a coin twice.
a) Calculate Var(Y).
b) Calculate Sd(Y).
Kevin Burke University of Limerick, Maths & Stats Dept 18 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Question 3
Consider the experiment of rolling two dice. Define the random
variable X = “the sum of the two numbers showing”.
a) Construct the probability distribution of X.
b) Calculate E(X).
c) Calculate E(X
2
).
d) Calculate Sd(X).
Kevin Burke University of Limerick, Maths & Stats Dept 19 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Moments of a Distribution
The moments of a distribution describe various aspects of its shape
as follows:
1st: E(X) ⇒ the mean.
2nd: E(X
2
) ⇒ required for variance.
3rd: E(X
3
) ⇒ required for skewness (tail pointing left/right).
4th: E(X
4
) ⇒ required for kurtosis (“heaviness” of tails).
.
.
.
kth: E(X
k
) ⇒ higher order aspects of its shape.
Kevin Burke University of Limerick, Maths & Stats Dept 20 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Moment Generating Function
Often we require a general procedure for generating moments without
having to carry out expectation each time.
Therefore, we define the moment generating function
MX (t) = E(e
X t) .
We will see that differentiating this function k times (with respect to t)
and then setting t = 0 produces the kth moment, i.e.,
d
k
dtk MX (0) = E(X
k
) .
Kevin Burke University of Limerick, Maths & Stats Dept 21 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Moment Generating Function
MX (t) = E(e
X t)
d
dt MX (t) = E(XeX t) ⇒ d
dt MX (0) = E(Xe0
) = E(X)
d
2
dt2 MX (t) = E(X
2e
X t) ⇒ d
2
dt2 MX (0) = E(X
2e
0
) = E(X
2
)
d
3
dt3 MX (t) = E(X
3e
X t) ⇒ d
3
dt3 MX (0) = E(X
3e
0
) = E(X
3
)
.
.
.
d
k
dtk MX (t) = E(X
k
e
X t) ⇒ d
k
dtk MX (0) = E(X
k
e
0
) = E(X
k
)
Often the process of deriving the moment generating function and
successively differentiating it is easier than calculating E(X
k
) directly.
Kevin Burke University of Limerick, Maths & Stats Dept 22 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Example: Flipping Two Coins
We continue with X = “the number of heads” where p(0) = 1
4
,
p(1) = 1
2 =
2
4
and p(2) = 1
4
. The moment generating function is then
MX (t) = E(e
X t) = e
0
(
1
4
) + e
t
(
2
4
) + e
2t
(
1
4
)

1
4
(1 + 2 e
t + e
2t
).
Differentiating this twice gives
d
dt MX (t) = 1
4
(2 e
t + 2 e
2t
),
d
2
dt2 MX (t) = 1
4
(2 e
t + 4 e
2t
),
and the first two moments are
E(X) = d
dt MX (0) = 1
4
(2 e
0 + 2 e
0
) = 4
4 = 1,
E(X
2
) = d
2
dt2 MX (0) = 1
4
(2 e
0 + 4 e
0
) = 6
4 = 1.5,
as previously calculated.
Kevin Burke University of Limerick, Maths & Stats Dept 23 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Question 4
We had the random variable Y = “the number of unique faces” based
on flipping a coin twice.
y 1 2
Pr(Y = y)
1
2
1
2
a) Derive the moment generating function.
b) Use the answer to part (a) to calculate E(X) and E(X
2
).
Kevin Burke University of Limerick, Maths & Stats Dept 24 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Discrete Versus Continuous
We have considered discrete random variables:
X takes values from a discrete set {x1, x2, x3, . . . , xn}.
The total probability is Pr(X ∈ {x1, x2, x3, . . . , xn}) = Pp(xi) = 1.
Here p(x) = Pr(X = x) is the probability mass function which
assigns a probability to each value.
Now consider continuous random variables:
X can take any value in an interval [a, b].
The total probability is Pr(X ∈ [a, b]) = 1.
We cannot assign a probability to each X ∈ [a, b] since there are an
infinite number of values - so what can we do?
Kevin Burke University of Limerick, Maths & Stats Dept 25 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Understanding Continuous Variables
To bridge the gap between discrete and continuous distributions, we
can split [a, b] into n − 1 discrete sub-intervals using n equally spaced
points: a = x1 < x2 < · · · < xn = b.
⇒ Pr(X ∈ [a, b] ) =
nX−1
i=1
Pr(X ∈ [xi
, xi+1] ) = 1.
Now assume that there exists a function f(x) which describes these
n − 1 probabilities as follows:
Pr(X ∈ [xi
, xi+1] ) = f(xi)∆x
Here ∆x = xi+1 − xi
is the distance between the equally spaced
points. Note: since Pr(X ∈ [xi
, xi+1] ) cannot be negative, f(x) ≥ 0.
Kevin Burke University of Limerick, Maths & Stats Dept 26 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Every Value in the Continuous Interval
Thus, the previous sum becomes
Pr(X ∈ [a, b] ) =
nX−1
i=1
f(xi)∆x = 1.
To incorporate every value in [a, b], we must increase n to infinity
producing an infinite number of sub-intervals of length ∆x ≈ 0.
Thus we get
Pr(X ∈ [a, b] ) = lim
n→∞
Xn−1
i=1
f(xi)∆x =
Z b
a
f(x) dx
| {z }
By definition of an integral
= 1.
Kevin Burke University of Limerick, Maths & Stats Dept 27 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Continuous: Probability Density Function
Probabilities are calculated (using integration) through a probability
density function which has the following properties:

1. f(x) ≥ 0 (probabilies cannot be negative)

Z b
a
f(x) dx = 1 (total probability equals one)
where X ∈ [a, b].
We call f(x) a “density” function to distinguish it from a mass function
p(x) which applies to discrete distributions.
Note that whereas p(x) = Pr(X = x) is a probability, f(x) is not a
probability but it produces probabilities through integration.
Kevin Burke University of Limerick, Maths & Stats Dept 28 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Replace Sums with Integrals
So for continuous random variables we have integrals (in contrast to
discrete where we have sums.)
All concepts introduced in this lecture still hold for continuous random
variables but sums are replaced with integrals, for example:
E(X) = Z b
a
x f(x) dx
E[g(X)] = Z b
a
g(x)f(x) dx
MX (t) = Z b
a
e
xt f(x) dx
Kevin Burke University of Limerick, Maths & Stats Dept 29 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Example: Continuous Distribution
Consider the a continuous random variable X ∈ [0, 6] with density
function f(x) = 1
18 x.
Z 6
0
1
18 x dx =
h
1
18
1
2
x
2
i6
0

h
1
36x
2
i6
0
= 1
36 (36) − 1
36 (0) = 1.
Therefore f(x) is a valid density function (it integrates to one).
The expected value is
E(X) = Z 6
0
x
1
18 x dx =
Z 6
0
1
18x
2 dx =
h
1
18
1
3
x
3
i6
0
= 4.
Kevin Burke University of Limerick, Maths & Stats Dept 30 / 31
Random Variables Expected Value Variance and Standard Deviation Moment Generating Function Continuous RVs
Example: Continuous Distribution
The moment generating function is
MX (t) = E(e
Xt) = Z 6
0
e
xt 1
18 x dx =
1
18 Z 6
0
e
xt x dx
Integration by parts:
u = x ⇒ du = dx dv = e
xtdx ⇒ v = 1
t
e
xt
⇒ MX (t) = 1
18 
x
1
t
e
xt −
Z
1
t
e
xtdx6
0

1
18
x
1
t
e
xt −
1
t
2 e
xt 6
0

1
18 h
6
1
t
e
6t −
1
t
2 e
6t +
1
t
2
i

1
18
e
6t
(6t − 1) + 1