Code

WIP Kron#843

Code

Draft

dlwh wants to merge 64 commits intostanford-crfm:mainstanford-crfm/levanter:mainfrom evanatyourservice:kronevanatyourservice/levanter:kronCopy head branch name to clipboard

+1,312-6

Commits

Commits on Jan 1, 2024

• logs

  

  Kaiyue Wen

  committed
  86d6556

• try dataset path

  WhenWen

  committed
  8b0900f

Commits on Oct 12, 2024

• use the jax serialization manager for deser in an attempt to fix crash. (it doesn't)

  dlwh

  committed
  5bf8cd0

Commits on Oct 13, 2024

• cleaner data loader but ti doesn't help :(

  dlwh

  committed
  20a4568

Commits on Oct 14, 2024

• ok this maybe fixed it?

  dlwh

  committed
  2747705

• cleanup

  dlwh

  committed
  538f0ed

• fix tests

  dlwh

  committed
  2c5ee4b

• fix what is probably the underlying problem

  dlwh

  committed
  ab543d6

• wip

  dlwh

  committed
  6395305

Commits on Dec 3, 2024

• Merge branch 'main' of github.com:WhenWen/levanter-2024 into main

  

  Kaiyue Wen

  committed
  b8f28f4

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  f98e376

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  e961c95

• wip

  dlwh

  committed
  2b80af7

• enough device puts and we're good

  dlwh

  committed
  87d7665

• ok we're good

  dlwh

  committed
  074d0ec

• Merge remote-tracking branch 'origin/main' into WhenWen/main

  dlwh

  committed
  5668289

• Merge branch 'use_manager_deser' into WhenWen/main

  dlwh

  committed
  d2d310e

• fix tree leaf stuff

  dlwh

  committed
  722edaf

Commits on Dec 4, 2024

• add map_flattened_linear_layers use in muon

  dlwh

  committed
  5692611

• Merge remote-tracking branch 'origin/main' into muon

  dlwh

  committed
  2f119bd

• adding kron file to optim

  evanatyourservice

  committed
  0f41ebb

Commits on Dec 5, 2024

• testing 123

  evanatyourservice

  committed
  fe3ecc9

• Update kron.py

  evanatyourservice

  committed
  37452c7

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  701956d

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  aac1cee

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  e44e7fa

Commits on Dec 6, 2024

• Update kron.py

  evanatyourservice

  committed
  476ba36

Commits on Dec 7, 2024

• expose precond lr and init

  evanatyourservice

  committed
  966b80e

• Merge branch 'stanford-crfm:main' into kron

  evanatyourservice

  authored
  6408d23

Commits on Dec 10, 2024

• Update kron.py

  evanatyourservice

  committed
  91e29e7

Commits on Dec 14, 2024

• Merge remote-tracking branch 'upstream/main' into kron

  evanatyourservice

  committed
  6b9ce26

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  53efaa3

• Update README.md

  evanatyourservice

  committed
  311da92

• Update kron.py

  evanatyourservice

  committed
  33c17f4

• Update kron.py

  evanatyourservice

  committed
  8da6e34

• Update kron.py

  evanatyourservice

  committed
  5607cec

Commits on Dec 15, 2024

• trust remote code

  evanatyourservice

  committed
  2fb6c34

• settings defaults

  evanatyourservice

  committed
  336e1e1

• no key, deterministic, pass all into cond, more sharding

  evanatyourservice

  committed
  a5ff351

• set key in state

  evanatyourservice

  committed
  3a06e1c

• whoops

  evanatyourservice

  committed
  f7f2382

• small fix

  evanatyourservice

  committed
  07781e6

• Update kron.py

  evanatyourservice

  committed
  9ef0869

• Update kron.py

  evanatyourservice

  committed
  ed50cce

• settings

  evanatyourservice

  committed
  1dc0f43

Commits on Dec 16, 2024

• small fix in init sharding

  evanatyourservice

  committed
  f1c1b38

• trying repl only

  evanatyourservice

  committed
  7a6f501

• Revert "trying repl only"

  Show description for 3473eed
  evanatyourservice

  committed
  3473eed

• trying while loop

  evanatyourservice

  committed
  7684702

Commits on Dec 19, 2024

• trying more simple psgd kron version

  evanatyourservice

  committed
  d284518

• Update kron.py

  evanatyourservice

  committed
  0c920b0

• Merge branch 'stanford-crfm:main' into kron

  evanatyourservice

  authored
  2feff32

• trying simple version

  evanatyourservice

  committed
  6a2e19f

• take out unavailable args

  evanatyourservice

  committed
  5108be0

• no extra args

  evanatyourservice

  committed
  975a2d7

• trying this

  evanatyourservice

  committed
  3fa70ab

• Update kron.py

  evanatyourservice

  committed
  b62963e

• Revert "Update kron.py"

  Show description for 5bafdcf
  evanatyourservice

  committed
  5bafdcf

• small fix

  evanatyourservice

  committed
  c47c4c5

• settings

  evanatyourservice

  committed
  ee747c0

Commits on Dec 22, 2024

• small changes/moving to remote

  evanatyourservice

  committed
  59f2c10

• Merge remote-tracking branch 'upstream/main' into kron

  evanatyourservice

  committed
  aa43e4f

• simplified kron is working, need to test on larger pod

  evanatyourservice

  committed
  25a2c20

Commits on Dec 23, 2024

• Update kron.py

  evanatyourservice

  committed
  88d49ed