## How is deep reinforcement learning used in continuous action spaces?

Deep Reinforcement Learning in Continuous Action Spaces Figure 1. The architecture of our policy-value network. As input, a feature map (Table 2 in the supplementary material) is provided from the state information.

**How does deep CNN work in continuous action spaces?**

Our deep CNN still discretizes the state space and the action space. However, in the stochastic continuous action search, we lift the restric- tion of the deterministic discretization and conduct a local search procedure in a physical simulator with continuous action samples.

**How are neural networks trained in discrete action spaces?**

Although Go has a ﬁnite, discrete action space, its depth of the play creates complex branches. Based on the moves of human experts, two neural networks in AlphaGo Lee are trained for the policy and value functions. AlphaGo Lee uses a Monte Carlo tree search (MCTS) for policy improvement.

### When to use policy based reinforcement learning ( RL )?

When the space is large, the usage of memory and computation consumption grows rapidly. The policy based RL avoids this because the objective is to learn a set of parameters that is far less than the space count.

**Why do we need large continuous action spaces?**

Learning good strategies from large continuous action spaces is important for many real-world problems includ- ing learning robotic manipulations and playing games with physical objects. In particular, when an autonomous agent interacts with physical objects, it is often necessary to han- dle large continuous action spaces.

**How is continuous action space used in curling?**

In the domain of curling, several algorithms have been pro- posed. As a way of dealing with continuous action space, game tree search methods (Yamamoto et al., 2015) have dis- cretized continuous action space. The evaluation functions are designed based on the domain knowledge and rules of the game.

## How is reinforcement learning used in simulated curling?

Deep Reinforcement Learning in Continuous Action Spaces: a Case Study in the Game of Simulated Curling Kyowoon Lee* 1Sol-A Kim Jaesik Choi1Seong-Whan Lee2 Abstract Many real-world applications of reinforcement learning require an agent to select optimal actions from continuous spaces.