Extracting attention matrix of each head per layer

[VarLad]

Hello! For an example like:

using Transformers
using Transformers.TextEncoders
using Transformers.HuggingFace

textencoder, bert_model = hgf"bert-base-uncased"

text1 = "Peter Piper picked a peck of pickled peppers"
text2 = "Fuzzy Wuzzy was a bear"

text = [[ text1, text2 ]] # 1 batch of contiguous sentences
sample = encode(textencoder, text)

bert_features = bert_model(sample)

Is there a way to get the attention matrix from every attention head for every layer for bert_features?

[VarLad]

Answered here already :)
#166