Skip to content

Conversation

@earlytobed
Copy link

This PR fixes #723, older PyTorch versions (pre-2.2, roughly, 7fc2929) don’t support arg generator, leading to the error:

Traceback (most recent call last):
    ...
    File "~/llm.c/train_gpt2.py", line 160, in _init_weights
    torch.nn.init.normal_(module.weight, mean=0.0, std=0.02, generator=self.init_rng)
TypeError: normal_() got an unexpected keyword argument 'generator'

after commit 86682af : torch.nn.init.normal_ called with generator, file: train_gpt2.py line146, line150

This PR replaces the torch.nn.init.normal_ with Tensor.normal_, ensuring compatibility with older PyTorch releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TypeError: normal_() got an unexpected keyword argument 'generator'

1 participant