Code

Implement Deep Q-Network#617

Code

Merged

BradLarson merged 36 commits intotensorflow:mastertensorflow/swift-models:masterfrom seungjaeryanlee:dqnseungjaeryanlee/swift-models:dqnCopy head branch name to clipboardAug 10, 2020

+546

Commits

Commits on Jun 29, 2020

• Copy code from Colab

  seungjaeryanlee

  committed
  5dc50eb

• Merge branch 'master' into dqn

  seungjaeryanlee

  committed
  9f049af

Commits on Jun 30, 2020

• Use .scalarized() to convert TF scalar to Swift

  seungjaeryanlee

  committed
  6268d58

• Improve code clarity

  seungjaeryanlee

  committed
  51d1fad

• Save isDone as Tensor<Bool>

  seungjaeryanlee

  committed
  84df320

• Save and use isDone for target calculation

  seungjaeryanlee

  committed
  2b99489

• Add commented parallelized training implementation

  seungjaeryanlee

  committed
  18e6294

• Save learning curve plot

  seungjaeryanlee

  committed
  36c1ddf

Commits on Jul 1, 2020

• Use parallelized training with custom gatherNd

  seungjaeryanlee

  committed
  da57062

• Add minBufferSize parameter

  seungjaeryanlee

  committed
  2ec956c

• Remove comments and refactor code

  seungjaeryanlee

  committed
  dab2a3f

• Fix bug where state was updated

  seungjaeryanlee

  committed
  0bc60ca

• Simplify code

  seungjaeryanlee

  committed
  01074d9

• Save TD loss curve

  seungjaeryanlee

  committed
  eca8a92

Commits on Jul 2, 2020

• Purge uses of _Raw operations

  seungjaeryanlee

  committed
  ae087dd

• Use Huber loss instead of MSE

  seungjaeryanlee

  committed
  4acd6ce

• Simplify Tensor initialization

  seungjaeryanlee

  committed
  22aaf75

• Set device explicitly on Tensor creation

  seungjaeryanlee

  committed
  24392f3

Commits on Aug 3, 2020

• Merge branch 'master' into dqn

  seungjaeryanlee

  committed
  441ab35

• Add minBufferSize to Agent argument

  seungjaeryanlee

  committed
  ccfa087

• Use soft target updates

  seungjaeryanlee

  committed
  65de04e

• Fix bug where isDone was used wrong

  seungjaeryanlee

  committed
  bcbb7e2

• Fix bug where target net is initialized with soft update

  seungjaeryanlee

  committed
  a203226

• Follow hyperparameters in swift-rl

  seungjaeryanlee

  committed
  e757c0f

• Run evaluation episode for every training episode

  seungjaeryanlee

  committed
  d2be5bd

Commits on Aug 4, 2020

• Implement combined experience replay

  seungjaeryanlee

  committed
  6a118ab

• Implement double DQN

  seungjaeryanlee

  committed
  ce539e5

• Add options to toggle CER and DDQN

  seungjaeryanlee

  committed
  cf7b96a

• Refactor code

  seungjaeryanlee

  committed
  98b4647

• Add updateTargetQNet to Agent class

  seungjaeryanlee

  committed
  e00901a

• Use TF-Agents hyperparameters

  seungjaeryanlee

  committed
  bca2614

Commits on Aug 5, 2020

• Changed ReplayBuffer to play better with GPU eager mode, restructured to four files, added Tensor extension, formatted via swift-format.

  BradLarson

  committed
  45b880e

Commits on Aug 6, 2020

• Fix ReplayBuffer pass-by-value bug

  seungjaeryanlee

  committed
  356c989

• Use epsilon decay for more consistent performance

  seungjaeryanlee

  committed
  d774fad

Commits on Aug 7, 2020

• Add documentation and improve names

  seungjaeryanlee

  committed
  a10f201

• Document Agent and ReplayBuffer parameters

  seungjaeryanlee

  committed
  4aa9296