Diagonal must be scalar in '/model/Trilu/' From 'QAicGraph', when compiling Sealion ONNX model to QPC format

[Ngyyen]

When attempting to compile a Sealion ONNX model (converted using optimum) to QPC format using the Qualcomm Cloud AI SDK's qaic-exec tool, I encounter an error related to the Trilu operation. The error message indicates that the diagonal parameter must be a scalar, causing the compilation to fail with a Function verification failed error.

Steps to Reproduce

1. Convert the Sealion model to ONNX format using optimum
   optimum-cli export onnx --model aisingapore/Llama-SEA-LION-v3.5-8B-R ./sealion-onnx-optimum/ --monolith

2. The result folder look like this:

3. Run the following qaic-exec command to compile the ONNX model to QPC format:

/opt/qti-aic/exec/qaic-exec \
-aic-hw \
-aic-hw-version=2.0 \
-compile-only \
-convert-to-fp16 \
-aic-num-cores=4 \
-m=sealion-onnx-optimum/model.onnx \
-onnx-define-symbol=batch_size,4 \
-onnx-define-symbol=sequence_length,1024 \
-aic-binary-dir=aic-binary-dir

Error Output

In '/model/Trilu/' From 'QAicGraph'
Diagonal must be scalar
For comparison `LHS Equal RHS` with:
LHS: 0
RHS: 1
In '/model/Trilu/' From 'QAicGraph'
Diagonal must be scalar
For comparison `LHS Equal RHS` with:
LHS: 0
RHS: 1
QAIC_ERROR: 
Error message: Function verification failed.
QAICException:Unable to AddNodesToGraphFromModel

Expected Behavior

The qaic-exec command should successfully compile the Sealion ONNX model to QPC format without errors, producing a valid QPC file in the specified aic-binary-dir.

Actual Behavior

The compilation fails due to the Trilu operation’s diagonal parameter not being a scalar, resulting in a Function verification failed error and an Unable to AddNodesToGraphFromModel exception.

[prohan86]

HI @Ngyyen,

As this is an LLM model, please refer to https://github.com/quic/cloud-ai-sdk/tree/1.19/tutorials/efficient_transformers