In the following questions assume:
• The matrix P(t)isa m!m probability transition-function matrix whose (i, j)th entry, pi,j (t),
is deﬁned to be P(X(t)= j|X(0) = i)foraCTMPhaving m ! m rate matrix !,whose
(i, j)th entry, !i,j,isdeﬁnedtobetherateoftransitionfromstate i to state j.Further,for
astationaryCTMPwealsohave:P(X(s+ t)= j|X(s)= i)=P(X(t)= j|X(0) = i).
• The expression e!t
is the matrix exponential of the matrix (! · t), where t is a scalar repre-
• The expression p(t)isthe1!m vector containing the time-t pmf of state probabilities, pi(t),
and of course pi(t) ” P(X(t)= i). (Alternately, you may prefer the notation “(t)forthis
vector pmf, which contains time-t pmf of state probabilities, “i(t)).
• The random variable Ti,j represents the time that the CTMP takes to make the transition
from state i to state j (with the understanding that that is the only transition possible).
• The random variable #i is the sojourn time for state i in a CTMP; that is, #i represents the
total time spent in the state i before making the next transition, regardless of which state
the system next transitions.
• In the context of both DTMCs and CTMPs, the symbol Bi refers to the bundle of state i,
where a bundle is the set of state indices for all of the states that can be visited directly (in
one step) from state i.
• The symbol |Bi| is the cardinality of the set Bi;thatis,thecountofthenumberofelements
in the set Bi.
CTMP Results from MRP Chapter 5, Section 5.11, Table 5.1 are repeated on the
next page. Observe that in this table the notation for derivative is the prime symbol.
1. (25 total points) For Question 1, consider the four-state CTMP that has rate diagram as
given, and measures time in hours. Initially the process is in State 3.
(a) (5 points) Write the corresponding state-transition rate matrix, “.
(b) (5 points) If the process has been in State 1 for
4.4 hours, write the mean length of
time before leaving State 1.
(c) (5 points) If the process has been in State 1 for
4.4 hours, state the probability that the
next state to be visited is State 4.
(d) (5 points) Compute P(X(1) = 1|X(0) = 1). (Remember to show your work, any
formulae used, and/or MATLAB code).