Regarding Visualize Attention

[spartian]

Where should I call visualize attention from? I am going through your code and have not been able to decide from where should I call it. Can you tell me from where can I call the visual attention code like in HATT.py?I have my own dataset and I am trying to visualize which words have caused the labels to be assigned?

[AlexGidiotis]

You should call the function utils.visualize_attention anywhere in your test code. Another way to do it would be to add a secondary output to the attention layer that returns the attention weights as well.

[spartian]

By test code, do you mean the code that needs to written for test like eval.py? or a particular section of code in hatt.py?

[happy-machine]

Hi Alex, thanks for this, brilliant to have an accessible self-attention layer for Keras!

I'm having issues with this function aswell, im passing it ([tokens],model,reverse_dict,10)

I'm passing the model (which is working fine with some slightly altered code) model.predict([[tokens]])
where tokens is an array of a test test converted to numeric tokens

I get:
TypeError: only integer scalar arrays can be converted to a scalar index

If i log [topkeys] i get an array of numbers, but they don't correspond to the test sequence either in terms of the tokens themselves or the index

ie. if [topkeys] returns [14, 25, 90]
the tokens array for example has a zero at one of those locations when using them as an index on the tokens array. If i assume that these are supposed to represent tokens, im finding not all of these tokens are contained in the tokens array

is [topkeys] supposed to return indexs or tokens?

[happy-machine]

Also Alex, i was wondering why you're using binary_crossentropy when normally you'd use categorical for multi label?