Megatron-LM

Merge branch 'interleaved-pipeline-bugfix' into 'main'

Perform grad sync at correct place in interleaved pipeline parallelism

Merge branch 'outputtensor_index' into 'main'

fix indexation for output tensor after gradscaler call

Merge branch 'lmcafee/rotary-kwarg-dev' into 'main'