bacow/easy-rl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update chapter4_questions&keywords.md

Browse Source
This commit is contained in:
Yiyuan Yang
2021-05-24 09:43:57 +08:00
committed by GitHub
parent 14a00afcc0
commit beafd08c46
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/chapter4/chapter4_questions&keywords.md
View File

@@ -101,9 +101,9 @@
$$ $$
带入第三个式子，可以将其化简为： 带入第三个式子，可以将其化简为：
$$\begin{aligned} $$\begin{aligned}
\nabla_{\theta}J(\theta) &=& E_{\tau \sim p_{\theta}(\tau)}[{\nabla}_{\theta}logp_{\theta}(\tau)r(\tau)] \\ \nabla_{\theta}J(\theta) &= E_{\tau \sim p_{\theta}(\tau)}[{\nabla}_{\theta}logp_{\theta}(\tau)r(\tau)] \\
&=& E_{\tau \sim p_{\theta}}[(\nabla_{\theta}log\pi_{\theta}(a_t|s_t))(\sum_{t=1}^Tr(s_t,a_t))] \\ &= E_{\tau \sim p_{\theta}}[(\nabla_{\theta}log\pi_{\theta}(a_t|s_t))(\sum_{t=1}^Tr(s_t,a_t))] \\
&=& \frac{1}{N}\sum_{i=1}^N[(\sum_{t=1}^T\nabla_{\theta}log \pi_{\theta}(a_{i,t}|s_{i,t}))(\sum_{t=1}^Nr(s_{i,t},a_{i,t}))] &= \frac{1}{N}\sum_{i=1}^N[(\sum_{t=1}^T\nabla_{\theta}log \pi_{\theta}(a_{i,t}|s_{i,t}))(\sum_{t=1}^Nr(s_{i,t},a_{i,t}))]
\end{aligned}$$ \end{aligned}$$
- 高冷的面试官：可以说一下你了解到的基于梯度策略的优化时的小技巧吗？ - 高冷的面试官：可以说一下你了解到的基于梯度策略的优化时的小技巧吗？