Questions tagged [temporal-difference]
The temporal-difference tag has no summary.
25 questions
1
vote
0
answers
38
views
Choosing a Reference Value when Releveling Factors to Calculate Change Over Time
Context: I have a data set based around 16 different locations. Each location has a contaminant value measured once per year, from 2012 to 2023. The data looks something like this:
Location
Type
Year
...
- 11
1
vote
0
answers
94
views
Include Time as a independent variable in the model for an ecological study
I am conducting my master's thesis on the temporal patterns of spore production in two specific species and the environmental drivers associated with these patterns. I began with a visual analysis of ...
1
vote
1
answer
187
views
Time series with autocorrelation and partial-autocorrelation
I am working with time-series data, where each day is represented by a CSV file containing a 24×25 grid, with each entry acting as a pixel.
I have generated ACF and PACF to understand the temporal ...
- 15
1
vote
1
answer
128
views
Correct binomial GLMM for temporal trends in species occurrences
I have a dataset with 80 species which were sampled in about 120 water bodies at two time periods (historical / recent). Only presence/absence of the species in each water body is considered. The data ...
- 11
1
vote
1
answer
68
views
Ecological temporal statistical analysis question
I collected animal samples without replacement over three time periods from the same locality: seven years apart in the deep-sea (eg, no known seasonality). I want to know whether the mean difference ...
- 11
2
votes
1
answer
141
views
Is Expected Sarsa is off-policy, and SARSA is just an MC estimate of Expected SARSA, why is it on-policy?
So, expected SARSA defines the update as:
$$
Q(s,a) = Q(s,a) +\alpha (R+ \mathbb{E}_{a\sim\pi(s')}[Q(s', a)] - Q(s,a))
$$
Where SARSA defines the update as $a'\sim\pi(s')$:
$$
Q(s,a) = Q(s,a) +\alpha (...
- 1,561
2
votes
0
answers
201
views
Tsitsiklis and Van Roy’s Counterexample - Reinforcement Learning Understanding Math Derivations
I have been going through "Sutton & Barto Book: Reinforcement Learning: An Introduction", and in "Chapter 11: Off-policy Methods with Approximation", Example 11.1 briefly ...
- 21
2
votes
1
answer
97
views
What are suitable statistical approaches to examine temporal variation of species abundance/ community composition across multiple sites?
I'm looking for appropriate types of analysis for a data set that contains counts of different crab species across 4 sites with 3 replicates per site (12 in total) over a time period of 1.5 years - 5 ...
- 21
1
vote
0
answers
58
views
investigating and testing temporally overlapping data points in R
This is a two-part question that first asks how to query some data I have in R, and secondly, asks what might be the appropriate statistical operations to test any perceived relationships between ...
- 153
0
votes
0
answers
53
views
Modeling Temporal-Interval Distance
I'm trying to define a model for comparing temporal intervals, and can't find an appropriate distance function that incorporates the different relations two temporal intervals might have, e.g. overlap,...
1
vote
0
answers
1k
views
Why is least squares temporal difference (LSTD) method more sample efficient compared to Temporal difference (TD) for value function approximation
I am trying to understand how LSTD works in value function approximation. I am reading the preliminaries of this paper. I sort of understand how the LSTD method differs from TD learning.
In TD ...
- 1,136
0
votes
1
answer
2k
views
What is temporal leakage?
So I've been trying to work out exactly what temporal leakage is for a while now and I'm getting nowhere.
I'm not necessarily looking to code or anything, I'm more so interested in what it actually is ...
1
vote
1
answer
2k
views
How to normalise changes that occur over time spans of different duration?
I want to compare changes in a variable that occur over time spans of different duration.
Here is a hypothetical example:
Precipitation in Region A decreased by 300 mm (from 1000 mm to 700 mm) over a ...
- 111
2
votes
1
answer
749
views
How to define number of states in reinforcement learning
I'm a robotic engineer who's relatively new to reinforcement learning and I want to try to do simple reinforcement learning on a robot to optimize its velocity. I am however having trouble with ...
- 21
6
votes
1
answer
2k
views
Why is temporal difference learning biased in reinforcement learning?
When I learn reinforcement learning from David Silver's online video, I saw "the objective of TD learning, $r_t + \gamma V(s_{t+1})$ is a biased target for learning value function. " I know the ...
- 2,103