---
author: "Your name here"
title: "Math 365 Exam 2"
output: pdf_document
---

Please answer each problem as clearly and completely as you can.  Do not discuss these problems with other students, or anyone else but me. You may use your textbook, lecture notes, class materials (including those posted on the website), and homework, but do not use other books, the internet, or any materials other than those directly associated with the course.  Please do feel free to ask me questions, either via email or coming by my office.  Show all work to demonstrate that you understand your answer. You may use R for any computations; submit either your R script or pdf knit from your R markdown.

Exam is due Wednesday April 6 at the beginning of class. Late submissions will be penalized by 10 points per day. You may submit by emailing me your R markdown pdf or as a combination of paper and R work.

## Problem 1 (20pt)

Suppose that a continuous-time Markov chain with state space $S=\{1,2,3,4\}$ has infinitesimal generator given by the following matrix
```{r}
A<-matrix(0,4,4)
A[1,1:3]<-c(-3,2,1)
A[2,1:3]<-c(1,-2,1)
A[3,1:4]<-c(1,1,-3,1)
A[4,2:4]<-c(1,0,-1)
A
```
(a) Find the equilibrium distribution $\bar{\pi}$.

(b) If $X_0=2$, what is the expected amount of time until it changes state for the first time?

(c) If $X_0=1$, what is the expected amount of time until the chain first hits state 3?

(d) If $X_0=4$, what is the expected amount of time until the chain first hits state 3?

## Problem 2 (20pt) 

Consider a birth-and-death process with $\lambda_n=\frac{1}{n+1}$ and $\mu_n=1$ (except $\mu_0=0$).  Show that the process is positive recurrent and give the stationary distribution. Fully simplify using the fact that $e^x=\sum_{k=0}^\infty{\frac{x^k}{k!}}$.

## Problem 3 (20pt)  

Find the optimal stopping strategy and resulting value function $v(x)$ (where $x$ is the value of the initial roll) for the following game: Roll a fair 6-sided die. If it comes up 1 you lose. If it comes up $k>1$, you can either take a payoff of $k^2$ or roll again.

## Problem 4 (20pt) 

Suppose $X_1, X_2, \dots$ are independent identically distributed random variables with mean $\mu>0$. Show that $M_n= X_1X_2\cdots X_n/\mu^n$ is a martingale with respect to $X_1, X_2, \dots, X_n$.

## Problem 5 (20pt)  

Consider a two-server queueing system where customers arrive according to a Poisson process with rate $\lambda$ and  service times are exponential with rate $\mu$. Moreover, suppose that arrivals finding both servers busy immediately depart (don't enter the system), whereas those finding at least one free server immediately enter service and then depart when their service is completed.

(a) If both servers are presently busy, what is the expected time until the next customer enters the system?

(b) If the system starts with no customers, find the expected time until both servers are busy.

## Bonus Problem (5pt extra credit)

Consider this simple game: Flip a fair coin twice. You win if you get two heads and lose otherwise. The probability of winning is 1/4. Design a game involving only flips of fair coins such that you have a 1/3 probability of winning.