Have you backed up the Megatron-LM version you use?

[EthanLI24]

Hi,
I'm try to use your code to pretrain llama2-7b, but I find that Megatron-LM have been update recently and some code like 'indexed_dataset' have been removed/changed in the latest code.
Do you have an copy of the Megatron-LM you use in this repo?

[MoFHeka]

Please just use LlamaModel = GPTModel. For now Megatron have fully support Llama.
I may release my new framework code in the future.