Question about the different replication result

[riddlevv]

Hello!

Thank you for your great work. Recently I have followed the code you provided in github and your hyperparameters to train the Magicoder. However, The results I reproduced are different from the model you provided in huggingface. I am sure that everything is the same as your paper.

Here's my result.

To clarify, I use my own evaluation code to evaluate the two models. But since the two models share the same evaluation code, I think it doesn't matter.

Best regards,
Shen