11SojourntimesFrom the previous frame we can conclude that the sojourn times Ti, T2, .. are independentandexponentiallydistributedTheindependencefollowsfromtheMarkovpropertyNTNU-TrondheimNorwegianLniversityof梦Science and Technologywww.ntnu.edu
11 Sojourn times From the previous frame we can conclude that the sojourn times ܶଵ, ܶଶ, . are independent and exponentially distributed. The independence follows from the Markov property
12Sojourn timesWe may construct a Markov process as a stochastic process having the properties that eachtimeitentersastatei:1.The amout of time T, the process spends in states i before making a transition into adifferentstate isexponentiallydistributedwiththerateαi.2.When the process leaves state i, it will next enter state j with some probability Pujwhere Zj=o Pu, = 1.jtiThe mean sojourn time in state iis thereforeE(T)aiNTNU-TrondheimNorwegianUniversityofScienceandTechnologywww.ntnu.edu
12 Sojourn times We may construct a Markov process as a stochastic process having the properties that each time it enters a state ݅: 1. The amout of time ܶ ෩ the process spends in states ݅ before making a transition into a different state is exponentially distributed with the rate ߙ. 2. When the process leaves state ݅, it will next enter state ݆ with some probability ܲ, where ∑ ܲ ൌ 1. ୀ ஷ The mean sojourn time in state ݅ is therefore ܶ ܧ ෩ ൌ 1 ߙ
13TransitionratesLet auj be defined by aij =αiPuj for alli+jSinceaijis the rate at whichthe process leaves state i,and Pu, is the probability that itgoestostatej,itfollowsthataiiistheratewheninstateithattheprocessmakesatransition intostatej.Wecall aiiasthetransitionratefromitoj.Since Zj+i Puj = 1, there isai7=0jtiNTNU-TrondheimNorwegianLUniversityofScienceandTechnologywww.ntnu.edu
13 Transition rates Let ܽ be defined by ܽ ൌ ߙ ܲ · for all ്݆݅ Since ܽ is the rate at which the process leaves state ݅, and ܲ is the probability that it goes to state ݆, it follows that ܽ is the rate when in state ݅ that the process makes a transition into state ݆. We call ܽ as the transition rate from ݅ to ݆. Since ∑ஷ ܲ ൌ 1, there is ܽൌ ߙ ୀ ஷ
14TransitionratematrixWe can deduceaand Piiwhen weknow aii for all i,j,and may thereforedefine theMarkovprocessbyspecifying(i)thestatespaceXand(ii)thetransitionratesaijforalli,j,inX.The second definition is often more naturaland will be our main approach in thefollowingWemayarrangethetransitionratesaiiasamatrix:aor/a0oa01airai0a11A：:aroar1arrWherethediagonalelementsareai=-αiaii=0NTNU-TrondheimjtiNorwegian University of梦Science and Technologywww.ntnu.edu
14 Transition rate matrix We can deduce ܽ and ܲ when we know ܽ for all ݅, ݆, and may therefore define the Markov process by specifying (i) the state space X and (ii) the transition rates ܽ for all ݅, ݆, in X. The second definition is often more natural and will be our main approach in the following. We may arrange the transition rates ܽ as a matrix: ൌۯ ܽ ܽଵ ܽଵ ܽଵଵ ⋯ ܽ ܽଵ ⋮ ⋱⋮ ܽ ܽଵ ⋯ ܽ Where the diagonal elements are ܽ ൌ െߙ ൌܽ ୀ ஷ
15Transitionratematrix/a0oaora01aira10a11A:...:ariaroarrObservethattheentries of rowiof thetransition ratematrixAarethetransitionrates outofstatei (forj,i).Wewill call themdepartureratesfromstatei.Therate-aii=α,isthetotaldepartureratefromstatei.Thesumoftheentries inrowiisalwaysequaltoo,forall iex.NTNU-Trondheim莎Norwegian University ofScience and Technologywww.ntnu.edu
15 Transition rate matrix ൌۯ ܽ ܽଵ ܽଵ ܽଵଵ ⋯ ܽ ܽଵ ⋮ ⋱⋮ ܽ ܽଵ ⋯ ܽ Observe that the entries of row ݅ of the transition rate matrix ۯ are the transition rates out of state ݅ (for ݆, ݅). We will call them departure rates from state ݅. The rate െܽൌ ߙ is the total departure rate from state ݅. The sum of the entries in row ݅ is always equal to 0, for all ݅ ∈ X