Markov Chain Chap 9 - 26

Symmetric Random walk

> Q <- matrix(data = 0, nrow = 99, ncol = 99)
> p <- 0.5
> q <- 0.5
> for (i in 1:98) Q[i, (i + 1)] <- p
> for (i in 2:99) Q[i, (i - 1)] <- q
> N <- solve(diag(99) - Q)
> C <- rep(1, 99)
> R <- matrix(data = NA, nrow = 99, ncol = 2)
> R[, 1] <- c(rep(0, 98), p)
> R[, 2] <- c(q, rep(0, 98))
> N1 <- N
> NC1 <- N %*% C
> NR1 <- N %*% R
> NR1[50, ]
[1] 0.5 0.5

This gives the probability that a random walk starting from 50 will get absorbed in to one of these states

> plot(1:99, NR1[, 1], col = "blue", type = "l", lwd = 2)

Thus the probability is x/N

Now consider asymmetric random walk

> Q <- matrix(data = 0, nrow = 99, ncol = 99)
> p <- 0.49
> q <- 0.51
> for (i in 1:98) Q[i, (i + 1)] <- p
> for (i in 2:99) Q[i, (i - 1)] <- q
> N <- solve(diag(99) - Q)
> C <- rep(1, 99)
> R <- matrix(data = NA, nrow = 99, ncol = 2)
> R[, 1] <- c(rep(0, 98), p)
> R[, 2] <- c(q, rep(0, 98))
> N1 <- N
> NC1 <- N %*% C
> NR1 <- N %*% R
> NR1[50, ]
[1] 0.1191749 0.8808251

This gives the probability that a random walk starting from 50 will get absorbed in to one of these states

> plot(1:99, NR1[, 1], col = "blue", type = "l", lwd = 2, xlim = c(1,
+     100), ylim = c(0, 1))
> par(new = T)
> plot(1:99, (1:99/99), col = "red", type = "l", lwd = 2, xlim = c(1,
+     100), ylim = c(0, 1))

For a small change in probability , the prob of absorbtion changed drastically