Unexpected key(s) in state_dict when loading the model for evaluation

[JessicaLopezEspejel]

Hello,
Thank you for this work, I find it very interesting.
I have an issue running the code of evaluation on the test set after training the model. I get this error when the program tries to load the weights of the checkpoint model

Traceback (most recent call last): File "/home/users/jlopez/codes/transabs/evaluate.py", line 184, in <module> main(args) File "/home/users/jlopez/codes/transabs/evaluate.py", line 131, in main dh_model.load_state_dict(state_dict) File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 769, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for LMModel: Unexpected key(s) in state_dict: "transformer.article_embed.weight", "transformer.summary_embed.weight".

I hope you can help me fix the problem soon. Thank you!

[JessicaLopezEspejel]

I fixed the problem by adding the parameter (--doc_model). However, I am facing another problem:

Namespace(afn='gelu', attn_pdrop=0.1, beam=0, bpe_path='model/vocab_40000.bpe', checkpoint='/home/data/jlopez/transabs/outputs/medical/full_pretrained/checkpoints/checkpoint_18737.pt', clf_pdrop=0.1, data_file='/home/data/jlopez/transabs/datasets/medical/jsonl/full/test_encoded.jsonl', decoding_strategy=0, desc=None, doc_model=True, embd_pdrop=0.1, encoder_path='model/encoder_bpe_40000.json', gen_dir='gen', gen_len=110, k=10, max_len=110, min_len=None, n_batch=1, n_ctx=512, n_embd=768, n_head=12, n_layer=12, resid_pdrop=0.1, save_file='./res.txt', seed=42, stop_words=None, subset=None, tgt_dir='tgt')
device cuda n_gpu 8
Loading dataset...
Loading model...
Loading weights...
`
0%| | 0/6658 [00:10<?, ?it/s]
Traceback (most recent call last):

File "/home/users/jlopez/codes/transabs/evaluate.py", line 166, in

main(args)

File "/home/users/jlopez/codes/transabs/evaluate.py", line 123, in main
evaluate_model(dh_model, test_loader, text_encoder, device, args.beam, args.gen_len, args.k, args.decoding_strategy, args.save_file, args.gen_dir, args.tgt_dir, args.max_len, stop_words, args)

File "/home/users/jlopez/codes/transabs/evaluate.py", line 47, in evaluate_model
src_strs, tgt_strs, gen_strs = generate_outputs(model, pad_seq, mask_seq, text_encoder, device, beam, gen_len, k, decoding_strategy, min_len=args.min_len)

File "/home/users/jlopez/codes/transabs/generate.py", line 18, in generate_outputs
outputs = model(pad_output, mask_output, text_encoder, device, beam=beam, gen_len=gen_len, k=k, decoding_strategy=decoding_strategy, generate=True, min_len=min_len)

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 143, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 153, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)

File "/home/users/jlopez/.conda/envs/transabs_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/users/jlopez/codes/transabs/model_pytorch.py", line 219, in forward
return self.generate(pad_output, mask_output, text_encoder, device, beam, gen_len, k, decoding_strategy, min_len=min_len)

File "/home/users/jlopez/codes/transabs/model_pytorch.py", line 386, in generate
generated_toks = self.sample(XMB, mask, classify_idx, text_encoder, gen_len, k, decoding_strategy, min_len=min_len)
File "/home/users/jlopez/codes/transabs/model_pytorch.py", line 295, in sample
XMB = self.append_batch(XMB, next_idx)

File "/home/users/jlopez/codes/transabs/model_pytorch.py", line 236, in append_batch
next_x = torch.cat((next_idx, next_pos), -1).unsqueeze(1)
RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 2 and 1 at /opt/conda/conda-bld/pytorch_1550813258230/work/aten/src/THC/generic/THCTensorMath.cu:74`

I'm running the project using Python 3.7.7 and pytorch 1.0.1

Thank you