Courses

Preliminary concepts from Probability Theory. General stochastic processes. Markov chains in discrete time. Transition probability table, Chapman-Kolmogov equations. First visit (or return) probabilities and distribution of the time instant of first visit (or return). Classification of states. Repeated states, transient states, absorbing periodic states, non-periodic states. Classes of communicating states. Closed sets of states. Normal form of the transition table. Limit state behavior, ergodic theorem, stationary distribution. Study of random walks. Bankruptcy of the gambler. Markov chains in continuous time. Generating table, Kolmogorov equations, limit state behavior. Poisson process, birth-death process. Introduction to queuing systems.