Single Agent Dynamics
Last updated on Oct 29, 2021
Introduction
Motivation
IO: role of market structure on equilibrium outcomes.
Dynamics: study the endogenous evolution of market structure.
- Supply side dynamics
- Irreversible investment
- Entry sunk costs
- Product repositioning costs
- Price adjustment costs
- Learning by doing
- Demand side dynamics
- Switching costs
- Durable or storable products
Bonus motivation: AI literature studies essentially the same set of problems with similar tools (Igami 2020)
- Irony: niche topic in IO (super niche in econ), but at the core of
the frontier in computer science
- Why? Computation is hard, estimation harder, but extremely powerful prediction tool
- The world is intrinsecally dynamic
Examples (1)
Some examples in empirical IO
- Investment
- Rust (1987): bus engine replacement decision
- Durable goods
- Gowrisankaran and Rysman (2012): consumer demand in the digital camcorder industry
- Stockpiling
- Learning
- Switching costs
- Handel (2013): inertia in demand for health insurance
Examples (2)
But also in other applied micro fields:
- Labor economics
- Should you go to college? (Keane and Wolpin 1997)
- Health economics
-
Which health insurance to pick given there are switching costs? (Handel 2013)
-
Addiction (Becker and Murphy 1988)
-
- Public finance
- How should you set optimal taxes in a dynamic environment? (Golosov et al. 2006)
Do we really need dynamics?
In some cases, we can reduce a dynamic problem to a:
- Static problem
- Reduced-form problem
E.g., Investment decision
-
Dynamic problem, as gains are realized after costs
-
“Static” solution: invest if
-
Action today (
or ) does not affect the amount of future payoffs (NPV)
But many cases where it’s hard to evaluate dynamic questions in a static/reduced-form setting.
- Typically, cases where decision today would affect payoffs tomorrow
- And you care about those payoffs (
myopia)
“A dynamic model can do anything a static model can.”
New Empirical IO
So-called New Empirical IO (summary in Bresnahan (1989))
- Some decisions today might affect payoffs tomorrow
- But the decision today depends on the state today
- And the state today might have been the result of a decision yesterday
- Etc…
- Need dynamics to study these questions
- Where does it all start?
Pros and Cons
Advantages
-
We can adress intertemporal trade-offs
- Flow vs stock stocks and benefits
-
We can examine transitions and not only steady states
-
We are able to address policy questions that cannot be addressed with reduced-form methods
- Standard advantage of structural estimation
- But in a context with relevant intertemporal trade-offs / decisions
Disadvantages
-
We typically need more assumptions
- Robustness testing will therefore be important
-
Identification in dynamic models is less transparent
- Thus time should be spent articulating what variation in the data identifies our parameters of interest)
-
It is often computationally intensive (i.e., slow / unfeasible)
From Statics to Dynamics
Typical steps
- Specify the primitives of the model
- Static: single period agents’ payoff functions (utility or profit)
- Dynamic: static payoffs + evolution of state variables
- Can be exogenous
- … or endogenous: decision today has an effect on the state tomorrow
- Solve for optimal behavior
- Static: tipically agents maximize current utility or profit
- Dynamic: agents maximize present discounted value of future utilities or profits
- Search for parameter values that result in the “best match” between our model predictions and observed behavior
1st year Macro Recap
Markov Decision Processes
Formally, a discrete-time MDP consists of the following objects
-
A discrete time index
, for -
A state space
-
An action space
- and a family of constraint sets
- and a family of constraint sets
-
A family of transition probabilities
-
A discount factor,
-
A family of single-period reward functions
- so that the utility functional
has an additively separable decomposition
- so that the utility functional
MDP (2)
In words
-
The state space
contains all the information needed to- compute static utilities
- compute transition probabilities
- compute static utilities
-
The (conditional) action space
contains all the actions available in state- How can it be different by state? E.g. entry/exit decision if you’re in/out of the market
-
The transition probabilities
define the probabilities of future states conditional on- Present state
- Present decision
- Present state
-
The discount factor
together with the static reward functions determines the objective function
Notation
Brief parenthesis on notation
-
I have seen states denoted as
(for state)- others, depending on the specific context, e.g.
for experience
I will try to stick to
all the time -
I have seen decisions denoted as
(for action) (for decision)- others, depending on the specific context, e.g.
for investment
I will try to stick to
all the time
Maximization Problem
The objective is to pick the decision rule (or policy function)
Stationarity
In many applications, we assume stationarity
-
The transition probabilities and utility functions do not directly depend on
- i.e., are the same for all
- i.e., are the same for all
-
Uncomfortable assumption?
-
You think there is some reason (variable) why today’s probabilities should be different from tomorrow’s?
- If observable, include that variable in the state space
- If unobservable, integrate it out
Stationarity (2)
-
In the finite horizon case (
), stationarity does not help much still depends on , conditional on- Why? Difference between
and matters in the sum
-
In infinite-horizon problems, stationarity helps a lot
-
Now the difference between
and is always the same, i.e. -
does not depend on , conditional on -
The future looks the same whether the agent is in state
at time or in state at time
-
Value Function
Consider a stationary infinite-horizon problem
-
The only variable which affects the agent’s view about the future is the current value of the state,
-
We can rewrite the agent’s problem as
where- The expectation is taken over future states
- that evolve according to
- that evolve according to
is called the value function
How to solve?
- One could try to solve it by brute force
- i.e. try to solve for the structure of all of the optimal
decisions,
- Indeed, for finite-horizon problems, that might be necessary
- i.e. try to solve for the structure of all of the optimal
decisions,
- For stationary infinite-horizon problems, the value and policy
function should be time invariant
- What do we gain?
Bellman Equation
Bellman Equation (2)
We have now a recursive formulation of the value function: the
Bellman Equation
- The Bellman Equation is a functional equation
- Has to be satisfied in every state
- Can be written as
- We are actually looking for a fixed point of
The decision rule that satisfies the Bellman Equation is called the
policy function
Contractions
Under regularity conditions
is jointly continuous and bounded in is a continuous correspondence
It is possible to show that
- Contraction Mapping Theorem: then
has a unique fixed point!
Solving for the Value Function
How do we actually do it in practice?
- For finite horizon MDPs: backward induction
- Start from the last period: static maximization problem
- Move backwards taking the future value as given
- For infinite horizon MDPs: different options
- value function iteration
- most common
- policy function iteration
- successive approximations
- value function iteration
Difference with 1st year Macro
So what’s going to be new here?
- Estimation: retrieve model primitives from observed behavior
- And related: uncertainty
- Strategic interaction: multiple agents taking dynamic decisions
- Next lecture
Rust (1987)
Setting
Rust (1987): An Empirical Model of Harold Zurcher
-
Harold Zurcher (HZ) is the city bus superintendant in Madison, WI
-
As bus engines get older, the probability of malfunctions increases
-
HZ decides when to replace old bus engines with new ones
- Optimal stopping / investment problem
-
Tradeoff
- Cost of a new engine (fixed, stock)
- Repair costs, because of engine failures (continuous, flow)
-
Do we care about Harold Zurcher?
- Obviously not (and neither did Rust), it’s a method paper
- But referee asked for an application
Data
Units of observation
- Rust observes 162 buses over time
Observables: for each bus, he sees
- monthly mileage (RHS, state variable)
- and whether the engine was replaced (LHS, choice variable),
- in a given month
Variation
- on average, bus engines were replaced every 5 years with over 200,000 elapsed miles
- considerable variation in the time and mileage at which replacement occurs
Idea
- Construct a (parametric) model which predicts the time and mileage at which engine replacement occurs
- Use the model predictions (conditional on parameter values) to
estimate parameters that “fit” the data
- predicted replacements, given mileage VS observed replacements, given mileage
- Ideally use the estimates to learn something new
- e.g. the correct dynamic demand curve for bus engine replacement
Static Alternative
What would you do otherwise?
- You observe replacement decisions
- … and replacement costs
Regress replacement decision on replacement costs
Problem
- Replacement benefits are a flow (lower maintenance costs)
- … while the cost is a stock
Outcome
- We expect the overestimate demand elasticity. Why?
- Overpredict substitutions at low costs
- and underpredict substitution at high cost
Model
Assumptions of the structural model
- State:
- engine accumulated mileage at time
- Note: “continuous” in the data but has to be discretized into bins
- engine accumulated mileage at time
- Action:
- replace engine at time
- replace engine at time
- State transitions:
- mileage
evolves exogenously according to a 1st-order Markov process - The transition function is the same for every bus.
- If HZ replaces in period
( ), then
- mileage
Model (2)
HZ static utility function (for a single bus)
: expected costs of operating a bus with mileage- including maintenance costs & social costs of breakdown
- We would expect
is the cost of replacement (i.e., a new engine)- Note that replacement occurs immediately
: expected current utility from operating a bus with mileage and making replacement decision
Model (3)
HZ objective function is to maximize the expected present discounted
sum of future utilities
- The expectation
is over future , which evolve according to Markov process is over future choices ,- because HZ will observe future states
before choosing future actions , this is a functional
- because HZ will observe future states
Notes
- This is for one bus (but multiple engines).
- HZ has an infinite horizon for his decision making
summarizes state at time , i.e., the expected value of future utilities only depends on
Bellman Equation
This (sequential) representation of HZ’s problem is very cumbersome to work with.
We can rewrite
Notes:
- Same
on both sides of equation because of infinite horizon - the future looks the same as the present for a given (i.e., it doesn’t matter where you are in time). - The expectation
is over the state-transition probabilities,
Order of Markow Process
Suppose for a moment that
- We need both
and in the state space (i.e., contains , too), - and the expectation is over the transition probability
Parenthesis: State Variables
Which variables should be state variables? I.e. should be included in the state space?
General rule for 1st order markow processes: variables need to
- define expected current payoff, and
- define expectations over next period state (i.e., distribution of
)
What do you do otherwise? Integrate them out! Examples
- Weather affects static utitilies but not transition probabilities
- More annoying to replace the engine if it rains
- Integration means: “compute expected utility of Harold Zurcher before he opens the window”
- Month of the year affects transition probabilities but not utilities
- Buses are used more in the winter
- Integration means: “compute average transition probabilities over months”
Note: you can always get the non-expected value function if you know the probability of raining or the transition probabilities by month
Policy Function
Along with this value function comes a corresponding policy (or
choice) function mapping the state
How would this compare with the optimal replacement mileage if HZ was myopic?
- Answer: HZ would wait until
for the replacement action
Solving the Model
Why do we want to solve for the value and policy functions?
- We want to know the agent’s optimal behavior and the equilibrium outcomes
- and be able to conduct comparative statics/dynamics (a.k.a. counterfactual simulations)
We have the Bellman Equation
Contraction Mapping Theorem:
Value Function Iteration
What does Blackwell’s Theorem allow us to do?
- Start with any arbitrary function
- Apply the mapping
to get - Apply again
- Continue applying
, and will converge to the unique fixed point of- i.e., the true value function
- i.e., the true value function
- Once we have
, it’s fairly trivial to compute the policy function- Static optimization problem (given
)
- Static optimization problem (given
This process is called value function iteration
How to Reconcile Model and Data?
Ideal Estimation Routine
- Pick a parameter value
- Solve value and policy function (inner loop)
- Match predicted choices with observed choices
- Find the parameter value
that best fits the data (outer loop)- Makes the observed choices “closest” to the predicted choices
- (or maximizes the likelihood of the observed choices)
Issue: model easily rejected by the data
-
The policy function takes the the form: replace iff
-
Can’t explain the coexistence of e.g. “a bus without replacement at 22K miles” and “another bus being replaced at 17K miles” in the data
-
We need some unobservables in the model to explain why observed choices do not exactly match predicted choices
Rust (1987) - Estimation
Uncertainty
How can we explain different replacement actions at different mileages in the data?
- Add other observables
- Add some stochastic element
But where? Two options
- Randomness in decisions
- I.e. “Harold Zurcher sometimes would like to replace the bus engine but he forgets”
- Probably still falsifiable
- Also need “Harold Zurcher sometimes would like not to replace but replacement happens” 🤔🤔🤔
- Randomness in the state
- Harold Zurcher knows something that we don’t
- He always makes the optimal decision but based on somethig we don’t observe
Unobservables
Rust uses the following utility specification:
- The
are components of utility of alternative that are observed by HZ but not by us, the econometrician.- E.g., the fact that an engine is running unusually smoothly given its mileage,
- or the fact that HZ is sick and doesn’t feel like replacing the engine this month
- Note: we have assumed addictive separability of
- The
s also affect HZ’s replacement decision are both observed and relevant part of the state space
Can we still solve the model? Can we estimate it?
Unobservables (2)
The Bellman Equation becomes
- The problem is not Markow anymore
- Is
correlated with ? How? - Is
correlated with ? And ? How?
- Is
- Dimension of the state space has increased
- From
points, to 🤯🤯 - Assuming all variables assume
values
- From
- Number of variables to integrate over to compute expectation
has increased- From one variable,
, to three,
- From one variable,
Assumptions
Rust makes 4 assumptions to make the problem tractable:
- First order Markow process of
- Conditional independence of
from and - Independence of
from - Logit distribution of
Assumption 1
A1: first-order markov process of
-
What it buys
and prior to current period are irrelevant
-
What it still allows:
- allows
to be correlated with
- allows
-
What are we assuming away
- Any sort of longer run dependence
- Does it matter? If yes, just re-consider what is one time period
- Or make the state space larger (as usual in Markow processes)
Assumption 1 - Implications
The Bellman Equation becomes
- Now the state is
- sufficient, because defines both current utility and (the expectation of) next-period state, under the first-order Markov assumption
is now analogous to- State space now is
- From
- From
- Now we could use value function iteration to solve the problem
- If
is continuous, it has to be discretised
- If
Assumption 1 - Issues
Open issues
-
Curse of dimensionality in the state space: (
)- Before, there were
points in state space (discrete values of ) - Now there are
: each for , ,- (Assuming we discretize all state variables into
values)
- (Assuming we discretize all state variables into
- Generally, number of points in state space (and thus computational time) increases exponentially in the number of variables
- Before, there were
-
Curse of dimensionality in the expected value:
- For each point in state space (at each iteration of the contraction mapping), need to compute
- Before, this was a 1-dimensional integral (or sum), now it’s 3-dimensional
-
Initial conditions
Assumption 2
A2: conditional independence of
-
What it buys
is independent of is independent of and , conditional on
-
What it still allows:
can be correlated across time, but only through the process
-
What are we assuming away
-
Any time of persistent heterogeneity
-
Does it matter? Easily yes
-
There are tons of applications where the unobservables are either fixed or correlated over time
- If fixed, there are methods to handle unobserved heterogeneity (i.e. bus “types”)
-
Assumption 2 - Implications
The Bellman Equation is
- Now
is noise that doesn’t affect the future- That is, conditional on
, is uncorrelated with
- That is, conditional on
Remeber: if
does not affect the future, it should’t be in the state space! How? Integrate it out.
Rust Shortcut: ASV
Rust: define the alternative-specific value function
-
is the present discounted value of not replacing, net of -
The state does not depend on
! -
What is the relationship with the value function?
-
We have a 1-to-1 mapping between
and !- If we have one, we can get the other
Rust Shortcut
Can we solve for
Yes! They have a recursive formulation
- Rust (1988) shows that it’s a joint contraction mapping
- Memo: the state space now is
- instead of
- Much smaller!
- instead of
- Lesson: any state variable that does not affect continuation values (the future) does not have to be in the “actual” state space
Assumption 2 - Implications
We can also split the expectation in the alternative-specific value
function
- Distribution of
has to be simulated - Distribution of
depends on
Assumption 3
A3: independence of
-
What it buys
not correlated with anything
-
What are we assuming away
- Some state-specific noise… probably irrelevant
-
Open Issues
- Distribution of
has to be simulated
- Distribution of
Assumption 4
A4:
-
What it buys
- Closed form solution for
- Closed form solution for
-
What are we assuming away
-
Different substitution patterns
-
Relevant? Maybe, if there are at least three options (here binary choice)
- As logit assumption in demand estimation
-
Logit magic 🧙🪄
where
Assumption 4 - Implications
The Bellman equation becomes
- We got fully rid of
!- How? With a lot of assumptions
Estimation
So far we have analysized how the 4 assumptions help solving the model.
- What about estimation?
Maximum Likelihood
- For a single bus, the likelihood function is
- i.e. probability of observed decisions
- and sequence of states
- conditional on the initial state
- and the parameter values
What is the impact of the 4 assumptions on the likelihood function?
Likelihood Function (A1)
A1: First order Markow process of
- We gain independence across time
- We can decompose the joint distribution in marginals across time
Likelihood Function (A2)
A2: independence of
-
We can decompose the joint distribution of
and into marginals -
can be estimated from the data- we’ll come back to it
-
for
we need the two remaining assumptions
Likelihood Function (A3)
A3: Independence of
- No need to condition on
- E.g. probability of replacement
- In words: same distribution of shocks in every state
Likelihood Function (A4)
A4: Logit distribution of
- E.g. probability of replacement becomes
- We have a closed form expression!
Likelihood Function
The final form of the likelihood function for one bus is
can be estimated from the data- given mileage
and investment decision , what are the observed frequencies of future states ? - does not have to depend on
- given mileage
depends on we know how to compute- given a value of
- solve by value function iteration
Likelihood Function (2)
Since we have may buses,
Estimation
Now we have all the pieces to estimate
Procedure
- Estimate the state transition probabilities
- Select a value of
- Init a choice-specific value function
- Apply the Bellman operator to compute
- Iterate until convergence to
(inner loop)
- Apply the Bellman operator to compute
- Compute the choice probabilities
- Compute the likelihood
- Iterate (2-5) until you are have found a (possibly global) minimum (outer loop)
Results
What do dynamics add?
- Static demand curve (
) is much more sensitive to the price of engine replacement. Why?- Compares present price with present savings
- If you compare present price with flow of future benefits, you are
less price sensitive
- More realistic

Extensions
Main limitation of Rust (1987): value function iteration
- Costly: has to be done for each parameter explored during optimization
- Particularly costly if the state space is large
Solutions
- Solve the model without solving a fixed point problem
- Hotz and Miller (1993)
- Solve the model and estimate the parameters at the same time
- Inner and outer loop in parallel
- Imai, Jain, and Ching (2009)
- Treat the estimation as a constrained optimization problem
- MPEC, as for demand
- Use off-the-shelf optimization algorithms
- Su and Judd (2012)
We’ll cover Hotz and Miller (1993) since at the core of the estimation of dynamic games.
Hotz & Miller (1993)
Motivation
Setting: Harold Zurcher problem
- same model
- same assumptions
- same notation
- same objective
Problem: computationally intense to do value function iteration
Can we solve the model without solving a fixed point problem?
Estimation in Rust
How did we estimate the model in Rust? Two main equations
-
Solve the Bellman equation of the alternative-specific value function
-
Compute the expected policy function
-
Maximize the likelihood function
Can we remove step 1?
Hotz & Miller Idea(s)
Idea 1: it would be great if we could start from something like
- No need to solve for the value function
- But we would still need a to solve a fixed point problem
- Back from the start? No
Idea 2: could replace the RHS element with a consistent estimate
Unclear? No problem, let’s go slowly step by step
Two Main Equations
-
Bellman equation
-
Expected policy function
Expected decision before the shocks
- Not the policy function
- The policy function maps
- The expected policy function maps
- The policy function maps
- Easier to work with: does not depend on the shocks
- Not a deterministic policy, but a stochastic one
Hotz & Miller - Idea 1
How do we get from the two equations
…. we could then substitute the first equation into the second …
But, easier to work with a different representation of the value function.
Expected Value Function
Recall Rust value function (not the alternative-specific
-
Value of being in state
without knowing the realization of the shock- “Value of Harold Zurcher before opening the window and seeing if it’s raining or not”
-
Analogous to the relationship between policy funciton and expected policy function
-
Note
- expectation of future value now is only over
can be solved via value function iteration as the operator on the RHS is a contraction
- expectation of future value now is only over
Representation Equivalence
Recall the alternative-specific value function of Rust
Relationship with the value function
Relationship with the expected value function
Goal
We switched from alternative-specific value function
- But the goal is the same
Go from this representation
**Note **: the
, and functions are different functions now.
Express EV in terms of EP (1)
First, let’s ged rid of one operator: the max operator
-
We are just substituting the
with the policy -
Important: we got rid of the
operator -
But we are still taking the expectation over
- Future states
- Shocks
- Future states
Express EV in terms of EP (2)
Now we get rid of another operator: the expectation over
is the summation over the next states is the transition probability (conditional on a particular choice)
so that the expected value function becomes
Express EV in terms of EP (3)
The previous equation, was defined at the state level
- system of
equations, 1 for each state (value of )
If we stack them, we can write them as
: matrix of transition probabilities from state to , given decision is the dot product operator (or element-wise matrix multiplication)
Express EV in terms of EP (4)
Now we have a system of
Tearing down notation to the bare minimum, we have
and finally we can solve for
Express EV in terms of EP (5)
What is
Let’s consider for example the expected value of the shock, conditional
on investment
- where
is Euler’s constant.
We again got rid of another
Express EV in terms of EP (6)
Now we can substitute it back and we have an equation which is just a
function of primitives
Or more compactly
First Equation
What is the first equation?
- Is the expected static payoff of choice
in each state, - … integrated over the choice probabilities,
- It’s a
vector
Unconditional transition probabilities:
- Are the transition probabilities conditional on a choice
for every present and future state, - … integrated over the choice probabilities,
- It’s a
matrix
Recap
We got our first equation
I.e.
What about the second equation
?
From V to P
In general, the expected probability of investment is
With the logit assumption, simplifies to
Now we have also the second equation!
Hotz & Miller - Idea 2
Idea 2: Replace
-
will converge to the true , because is converging to asymptotically.- Note: pay attention to
vs here: does not generally converge to for arbitrary , because is converging to but not with any .
- Note: pay attention to
How to compute
-
From the data, you observe states and decisions
-
You can compute frequency of decisions given states
- In Rust: frequency of engine replacement, given a mileage (discretized)
-
Assumption: you have enough data
- What if a state is not realised?
- Use frequencies in observed states to extrapolate frequencies in unobserved states
Recap
Steps so far
-
Estimate the conditional choice probabilities
from the data- Nonparametrically: frequency of each decision in each state
-
Solve for the expected value function with the inverstion step
-
Compute the predicted CCP, given
What now? Use the estimated CCP to build an objective function.
Objective Function
We have (at least) 2 options
- Hotz and Miller (1993) use GMM
- Aguirregabiria and Mira (2002)
use MLE
- by putting
in the likelihood function instead of
- by putting
We will follow the second approach
Pseudo-Likelihood
The likelihood function for one bus is
- CCPs
: estimated from data - transition matrix
: estimated from the data, given - static payoffs
: known, given - discount factor
: assumed
Why pseudo-likelihood? We have inputed something that is not a primitive but a consistent estimate of an equilibrium object,
Comments
Now a few comments on Hotz and Miller (1993)
- Computational bottleneck
- Aguirregabiria and Mira (2002)
- Importance of the T1EV assumption
- Data requirements
- Unobserved heterogeneity
- Identification
Computational Bottleneck
There is still 1 computational bottleneck in HM: the inversion step
- With large state space, hard to invert
- Even with modern computational power
- Hotz et al. (1994): forward simulation of
the value function
- You have the policy, the transitions and the utilities
- Just compute discounted flow of payoffs
- Core idea behind the estimation of dynamic games
Aguirregabiria, Mira (2002)
Hotz and Miller (1993) inversion gets us a recursive equation in probability space
- instead of the Bellman Equation in the value space
Idea
- Do you gain something by iterating
imes? - Monte Carlo simulations: finite sample properties of K−stage
estimators improve monotonically with K
- But especially for
! - Really worth iterating once
- But especially for
Type 1 EV errors
Crucial assumption
- Without logit errors, we need to simulate their distribution
- True also for Rust
- But it’s generally accepted
- doesn’t imply it’s innocuous
Data Requirements
For both Hotz et al. (1994) and Rust (1987), we need to discretize the state space
- Can be complicated with continuous variables
- Problem also in Rust
- But particularly problematic in Hotz et al.
(1994)
- Relies crucially on consistency of CCP estimates
- Need sufficient variation in actions for each state
Unobserved Heterogeneity
Hotz et al. (1994) cannot handle unobserved heterogeneity or “unobserved state variables” that are persistent over time.
Example
-
Suppose there are 2 bus types
: high and low quality -
We don’t know the share of types in the data
-
With Rust
-
Parametrize the effect of the difference in qualities
- E.g. high quality engines break less often
-
Parametrize the proportion of high quality buses
-
Solve the value function by type
-
Integrate over types when computing choice probabilities
-
Unobserved Heterogeneity (2)
What is the problem with Hotz et al. (1994)?
-
The unobserved heterogeneity generates persistency in choices
- I don’t replace today because it’s high quality, but I also probably don’t replace tomorrow either
- Decisions independent across time only conditional on type
-
Likelihood of decisions must be integrated over types
-
Hotz & Miller needs consistent estimates of
-
Difficult when
is not observed!
Identification
Work on identification
- Rust (1994) and Magnac and Thesmar
(2002)
- Rust (1987) is non-paramentrically
underidentified
parametric assumptions are essential
- Rust (1987) is non-paramentrically
underidentified
- Aguirregabiria and Suzuki (2014)
- Kalouptsidi, Scott, and Souza-Rodrigues (2017)
- Abbring and Daljord (2020)
- Can identify discount factor with some “instrument” that shifts future utilities but not current payoff
- Kalouptsidi et al. (2020)
Appendix
References [references]
Abbring, Jaap H, and Øystein Daljord. 2020. “Identifying the Discount Factor in Dynamic Discrete Choice Models.” Quantitative Economics 11 (2): 471–501.
Aguirregabiria, Victor, and Pedro Mira. 2002. “Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models.” Econometrica 70 (4): 1519–43.
Aguirregabiria, Victor, and Junichi Suzuki. 2014. “Identification and Counterfactuals in Dynamic Models of Market Entry and Exit.” Quantitative Marketing and Economics 12 (3): 267–304.
Becker, Gary S, and Kevin M Murphy. 1988. “A Theory of Rational Addiction.” Journal of Political Economy 96 (4): 675–700.
Berry, Steven T. 1992. “Estimation of a Model of Entry in the Airline Industry.” Econometrica: Journal of the Econometric Society, 889–917.
Bresnahan, Timothy F. 1989. “Empirical Studies of Industries with Market Power.” Handbook of Industrial Organization 2: 1011–57.
Crawford, Gregory S, and Matthew Shum. 2005. “Uncertainty and Learning in Pharmaceutical Demand.” Econometrica 73 (4): 1137–73.
Erdem, Tülin, Susumu Imai, and Michael P Keane. 2003. “Brand and Quantity Choice Dynamics Under Price Uncertainty.” Quantitative Marketing and Economics 1 (1): 5–64.
Erdem, Tülin, and Michael P Keane. 1996. “Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets.” Marketing Science 15 (1): 1–20.
Golosov, Mikhail, Aleh Tsyvinski, Ivan Werning, Peter Diamond, and Kenneth L Judd. 2006. “New Dynamic Public Finance: A User’s Guide [with Comments and Discussion].” NBER Macroeconomics Annual 21: 317–87.
Gowrisankaran, Gautam, and Marc Rysman. 2012. “Dynamics of Consumer Demand for New Durable Goods.” Journal of Political Economy 120 (6): 1173–1219.
Handel, Benjamin R. 2013. “Adverse Selection and Inertia in Health Insurance Markets: When Nudging Hurts.” American Economic Review 103 (7): 2643–82.
Hendel, Igal, and Aviv Nevo. 2006. “Measuring the Implications of Sales and Consumer Inventory Behavior.” Econometrica 74 (6): 1637–73.
Hotz, V Joseph, and Robert A Miller. 1993. “Conditional Choice Probabilities and the Estimation of Dynamic Models.” The Review of Economic Studies 60 (3): 497–529.
Hotz, V Joseph, Robert A Miller, Seth Sanders, and Jeffrey Smith. 1994. “A Simulation Estimator for Dynamic Models of Discrete Choice.” The Review of Economic Studies 61 (2): 265–89.
Igami, Mitsuru. 2020. “Artificial Intelligence as Structural Estimation: Deep Blue, Bonanza, and AlphaGo.” The Econometrics Journal 23 (3): S1–24.
Imai, Susumu, Neelam Jain, and Andrew Ching. 2009. “Bayesian Estimation of Dynamic Discrete Choice Models.” Econometrica 77 (6): 1865–99.
Kalouptsidi, Myrto, Yuichi Kitamura, Lucas Lima, and Eduardo A Souza-Rodrigues. 2020. “Partial Identification and Inference for Dynamic Models and Counterfactuals.” National Bureau of Economic Research.
Kalouptsidi, Myrto, Paul T Scott, and Eduardo Souza-Rodrigues. 2017. “On the Non-Identification of Counterfactuals in Dynamic Discrete Games.” International Journal of Industrial Organization 50: 362–71.
Keane, Michael P, and Kenneth I Wolpin. 1997. “The Career Decisions of Young Men.” Journal of Political Economy 105 (3): 473–522.
Magnac, Thierry, and David Thesmar. 2002. “Identifying Dynamic Discrete Decision Processes.” Econometrica 70 (2): 801–16.
Pakes, Ariel. 1986. “Patents as Options: Some Estimates of the Value of Holding European Patent Stocks.” Econometrica 54 (4): 755–84.
Rust, John. 1987. “Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher.” Econometrica: Journal of the Econometric Society, 999–1033.
———. 1988. “Maximum Likelihood Estimation of Discrete Control Processes.” SIAM Journal on Control and Optimization 26 (5): 1006–24.
———. 1994. “Structural Estimation of Markov Decision Processes.” Handbook of Econometrics 4: 3081–3143.
Su, Che-Lin, and Kenneth L Judd. 2012. “Constrained Optimization Approaches to Estimation of Structural Models.” Econometrica 80 (5): 2213–30.