MLIR

Multi-Level IR Compiler Framework

'arm_sve' Dialect

Basic dialect to target Arm SVE architectures

This dialect contains the definitions necessary to target specific Arm SVE scalable vector operations.

Operations 

source

arm_sve.convert_from_svbool (arm_sve::ConvertFromSvboolOp) 

Convert a svbool type to a SVE predicate type

Syntax:

operation ::= `arm_sve.convert_from_svbool` $source attr-dict `:` type($result)

Converts svbool types (vector<[16]xi1> or vectors of that type, e.g. vector<2x3x[16]xi1>) to SVE predicate types. Note: Only the trailing dimension can be scalable.

Example 1: Convert a 1-D svbool mask to a SVE predicate.

%source = vector.load %memref[%c0] : memref<?xi1>, vector<[16]xi1>
%result = arm_sve.convert_from_svbool %source : vector<[4]xi1>

Example 2: Convert a 2-D svbool mask to a mask of SVE predicates.

%source = vector.load %memref[%c0, %c0] : memref<2x?xi1>, vector<2x[16]xi1>
%result = arm_sve.convert_from_svbool %source : vector<2x[8]xi1>

A svbool is the smallest SVE predicate type that has a in-memory representation (and maps to a full predicate register). In MLIR svbool is represented as vector<[16]xi1>. Smaller SVE predicate types (vector<[1|2|4|8]xi1>) must be stored as a svbool then converted back to the original predicate type after loading.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
sourcetrailing scalable vector of 1-bit signless integer values with dim -1 having a size of {16}

Results: 

ResultDescription
resulttrailing scalable vector of 1-bit signless integer values with dim -1 having a size of {16, 8, 4, 2, 1}

arm_sve.convert_to_svbool (arm_sve::ConvertToSvboolOp) 

Convert a SVE predicate type to a svbool type

Syntax:

operation ::= `arm_sve.convert_to_svbool` $source attr-dict `:` type($source)

Converts SVE predicate types (or vectors of predicate types, e.g. vector<4x[4]xi1>) to svbool types. Note: Only the trailing dimension can be scalable.

Example 1: Convert a 1-D SVE predicate to a svbool mask.

%source = vector.create_mask %dim_size : vector<[4]xi1>
%result = arm_sve.convert_to_svbool %source : vector<[4]xi1>
// => Results in vector<[16]xi1>

Example 2: Convert a 2-D mask of SVE predicates to a svbool mask.

%source = vector.create_mask %c2, %dim_size : vector<2x[2]xi1>
%result = arm_sve.convert_to_svbool %source : vector<2x[2]xi1>
// => Results in vector<2x[16]xi1>

A svbool is the smallest SVE predicate type that has a in-memory representation (and maps to a full predicate register). In MLIR svbool is represented as vector<[16]xi1>. Smaller SVE predicate types (vector<[1|2|4|8]xi1>) must be converted to a svbool before they can be stored.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
sourcetrailing scalable vector of 1-bit signless integer values with dim -1 having a size of {16, 8, 4, 2, 1}

Results: 

ResultDescription
resulttrailing scalable vector of 1-bit signless integer values with dim -1 having a size of {16}

arm_sve.intr.add (arm_sve::ScalableMaskedAddIIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.convert.from.svbool (arm_sve::ConvertFromSvboolIntrOp) 

Operands: 

OperandDescription
svboolvector<[16]xi1>

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.convert.to.svbool (arm_sve::ConvertToSvboolIntrOp) 

Operands: 

OperandDescription
maskvector<[1]xi1>, vector<[2]xi1>, vector<[4]xi1>, vector<[8]xi1>, or vector<[16]xi1>

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.fadd (arm_sve::ScalableMaskedAddFIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.fdiv (arm_sve::ScalableMaskedDivFIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.fmul (arm_sve::ScalableMaskedMulFIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.fsub (arm_sve::ScalableMaskedSubFIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.mul (arm_sve::ScalableMaskedMulIIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.psel (arm_sve::PselIntrOp) 

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
p1vector<[16]xi1>
p2vector<[1]xi1>, vector<[2]xi1>, vector<[4]xi1>, vector<[8]xi1>, or vector<[16]xi1>
index32-bit signless integer

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.sdiv (arm_sve::ScalableMaskedSDivIIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.sdot (arm_sve::SdotIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.smmla (arm_sve::SmmlaIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.sub (arm_sve::ScalableMaskedSubIIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.udiv (arm_sve::ScalableMaskedUDivIIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.udot (arm_sve::UdotIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.ummla (arm_sve::UmmlaIntrOp) 

Operands: 

OperandDescription
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values
«unnamed»scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.whilelt (arm_sve::WhileLTIntrOp) 

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
base64-bit signless integer
n64-bit signless integer

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.zip.x2 (arm_sve::ZipX2IntrOp) 

Operands: 

OperandDescription
v1scalable vector of any type values
v2scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.intr.zip.x4 (arm_sve::ZipX4IntrOp) 

Operands: 

OperandDescription
v1scalable vector of any type values
v2scalable vector of any type values
v3scalable vector of any type values
v4scalable vector of any type values

Results: 

ResultDescription
resLLVM dialect-compatible type

arm_sve.masked.addf (arm_sve::ScalableMaskedAddFOp) 

Masked addition for scalable vectors of floats

Syntax:

operation ::= `arm_sve.masked.addf` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.addf operation takes one scalable vector mask and two scalable vector operands, and perform floating point addition on active lanes. Inactive lanes will keep the value of the first operand.

Traits: Commutative

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of floating-point values
src2scalable vector of floating-point values

Results: 

ResultDescription
resscalable vector of floating-point values

arm_sve.masked.addi (arm_sve::ScalableMaskedAddIOp) 

Masked addition for scalable vectors of integers

Syntax:

operation ::= `arm_sve.masked.addi` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.addi operation takes one scalable vector mask and two scalable vector operands, and perform integer addition on active lanes. Inactive lanes will keep the value of the first operand.

Traits: Commutative

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values
src2scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

Results: 

ResultDescription
resscalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

arm_sve.masked.divf (arm_sve::ScalableMaskedDivFOp) 

Masked division for scalable vectors of floats

Syntax:

operation ::= `arm_sve.masked.divf` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.divf operation takes one scalable vector mask and two scalable vector operands, and perform floating point division on active lanes. Inactive lanes will keep the value of the first operand.

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of floating-point values
src2scalable vector of floating-point values

Results: 

ResultDescription
resscalable vector of floating-point values

arm_sve.masked.divi_signed (arm_sve::ScalableMaskedSDivIOp) 

Masked signed division for scalable vectors of integers

Syntax:

operation ::= `arm_sve.masked.divi_signed` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.divi_signed operation takes one scalable vector mask and two scalable vector operands, and perform integer signed division on active lanes. Inactive lanes will keep the value of the first operand.

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values
src2scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

Results: 

ResultDescription
resscalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

arm_sve.masked.divi_unsigned (arm_sve::ScalableMaskedUDivIOp) 

Masked unsigned division for scalable vectors of integers

Syntax:

operation ::= `arm_sve.masked.divi_unsigned` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.divi_unsigned operation takes one scalable vector mask and two scalable vector operands, and perform integer unsigned division on active lanes. Inactive lanes will keep the value of the first operand.

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values
src2scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

Results: 

ResultDescription
resscalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

arm_sve.masked.mulf (arm_sve::ScalableMaskedMulFOp) 

Masked multiplication for scalable vectors of floats

Syntax:

operation ::= `arm_sve.masked.mulf` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.mulf operation takes one scalable vector mask and two scalable vector operands, and perform floating point multiplication on active lanes. Inactive lanes will keep the value of the first operand.

Traits: Commutative

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of floating-point values
src2scalable vector of floating-point values

Results: 

ResultDescription
resscalable vector of floating-point values

arm_sve.masked.muli (arm_sve::ScalableMaskedMulIOp) 

Masked multiplication for scalable vectors of integers

Syntax:

operation ::= `arm_sve.masked.muli` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.muli operation takes one scalable vector mask and two scalable vector operands, and perform integer multiplication on active lanes. Inactive lanes will keep the value of the first operand.

Traits: Commutative

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values
src2scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

Results: 

ResultDescription
resscalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

arm_sve.masked.subf (arm_sve::ScalableMaskedSubFOp) 

Masked subtraction for scalable vectors of floats

Syntax:

operation ::= `arm_sve.masked.subf` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.subf operation takes one scalable vector mask and two scalable vector operands, and perform floating point subtraction on active lanes. Inactive lanes will keep the value of the first operand.

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of floating-point values
src2scalable vector of floating-point values

Results: 

ResultDescription
resscalable vector of floating-point values

arm_sve.masked.subi (arm_sve::ScalableMaskedSubIOp) 

Masked subtraction for scalable vectors of integers

Syntax:

operation ::= `arm_sve.masked.subi` $mask `,` $src1 `,` $src2 attr-dict `:` type($mask) `,` type($res)

The arm_sve.masked.subi operation takes one scalable vector mask and two scalable vector operands, and perform integer subtraction on active lanes. Inactive lanes will keep the value of the first operand.

Operands: 

OperandDescription
maskscalable vector of 1-bit signless integer values
src1scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values
src2scalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

Results: 

ResultDescription
resscalable vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values

arm_sve.psel (arm_sve::PselOp) 

Predicate select

Syntax:

operation ::= `arm_sve.psel` $p1 `,` $p2 `[` $index `]` attr-dict `:` type($p1) `,` type($p2)

This operation returns the input predicate p1 or an all-false predicate based on the bit at p2[index]. Informally, the semantics are:

if p2[index % num_elements(p2)] == 1:
  return p1 : type(p1)
return all-false : type(p1)

Example:

// Note: p1 and p2 can have different sizes.
%pd = arm_sve.psel %p1, %p2[%index] : vector<[4]xi1>, vector<[8]xi1>

Note: This requires SME or SVE2.1 (+sme or +sve2p1 in LLVM target features).

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
p1vector<[1]xi1>, vector<[2]xi1>, vector<[4]xi1>, vector<[8]xi1>, or vector<[16]xi1>
p2vector<[1]xi1>, vector<[2]xi1>, vector<[4]xi1>, vector<[8]xi1>, or vector<[16]xi1>
indexindex

Results: 

ResultDescription
resultvector<[1]xi1>, vector<[2]xi1>, vector<[4]xi1>, vector<[8]xi1>, or vector<[16]xi1>

arm_sve.sdot (arm_sve::SdotOp) 

Vector-vector dot product and accumulate op

Syntax:

operation ::= `arm_sve.sdot` $acc `,` $src1 `,` $src2 attr-dict `:` type($src1) `to` type($dst)

SDOT: Signed integer addition of dot product.

This function maps to the SDOT instruction, and it takes signless integer operands that the operation interprets as signed. It partitions the second and third vector inputs into groups of four elements. They calculate the dot product of each group (without loss of precision) and then add each result to the overlapping element of the first vector input.

Source: https://developer.arm.com/documentation/100987/0000

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
accscalable vector of 32-bit signless integer or 64-bit signless integer values of length 4/2
src1scalable vector of 8-bit signless integer or 16-bit signless integer values of length 16/8
src2scalable vector of 8-bit signless integer or 16-bit signless integer values of length 16/8

Results: 

ResultDescription
dstscalable vector of 32-bit signless integer or 64-bit signless integer values of length 4/2

arm_sve.smmla (arm_sve::SmmlaOp) 

Matrix-matrix multiply and accumulate op

Syntax:

operation ::= `arm_sve.smmla` $acc `,` $src1 `,` $src2 attr-dict `:` type($src1) `to` type($dst)

SMMLA: Signed integer matrix multiply-accumulate.

This function maps to the SMMLA instruction, and it takes signless integer operands that the operation interprets as signed. It partitions the inputs into 128-bit quadwords, with the first input containing a row-by-row 2×2 matrix of 32-bit integers, the second input containing a row-by-row 2×8 matrix of 8-bit integers, and the third input containing a column-by-column 8×2 matrix of 8-bit integers. For each quadword, they multiply the second input matrix by the third input matrix using natural arithmetic and then add the result to the first input using modular arithmetic.

Source: https://developer.arm.com/documentation/100987/0000

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
accscalable vector of 32-bit signless integer values of length 4
src1scalable vector of 8-bit signless integer values of length 16
src2scalable vector of 8-bit signless integer values of length 16

Results: 

ResultDescription
dstscalable vector of 32-bit signless integer values of length 4

arm_sve.udot (arm_sve::UdotOp) 

Vector-vector dot product and accumulate op

Syntax:

operation ::= `arm_sve.udot` $acc `,` $src1 `,` $src2 attr-dict `:` type($src1) `to` type($dst)

UDOT: Unsigned integer addition of dot product.

This function maps to the UDOT instruction, and it takes signless integer operands that the operation interprets as unsigned. It partitions the second and third vector inputs into groups of four elements. They calculate the dot product of each group (without loss of precision) and then add each result to the overlapping element of the first vector input.

Source: https://developer.arm.com/documentation/100987/0000

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
accscalable vector of 32-bit signless integer or 64-bit signless integer values of length 4/2
src1scalable vector of 8-bit signless integer or 16-bit signless integer values of length 16/8
src2scalable vector of 8-bit signless integer or 16-bit signless integer values of length 16/8

Results: 

ResultDescription
dstscalable vector of 32-bit signless integer or 64-bit signless integer values of length 4/2

arm_sve.ummla (arm_sve::UmmlaOp) 

Matrix-matrix multiply and accumulate op

Syntax:

operation ::= `arm_sve.ummla` $acc `,` $src1 `,` $src2 attr-dict `:` type($src1) `to` type($dst)

UMMLA: Unsigned integer matrix multiply-accumulate.

This function maps to the UMMLA instruction, and it takes signless integer operands that the operation interprets as unsigned. It partitions the inputs into 128-bit quadwords, with the first input containing a row-by-row 2×2 matrix of 32-bit integers, the second input containing a row-by-row 2×8 matrix of 8-bit integers, and the third input containing a column-by-column 8×2 matrix of 8-bit integers. For each quadword, they multiply the second input matrix by the third input matrix using natural arithmetic and then add the result to the first input using modular arithmetic.

Source: https://developer.arm.com/documentation/100987/0000

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
accscalable vector of 32-bit signless integer values of length 4
src1scalable vector of 8-bit signless integer values of length 16
src2scalable vector of 8-bit signless integer values of length 16

Results: 

ResultDescription
dstscalable vector of 32-bit signless integer values of length 4

arm_sve.zip.x2 (arm_sve::ZipX2Op) 

Multi-vector two-way zip op

Syntax:

operation ::= `arm_sve.zip.x2` $sourceV1 `,` $sourceV2 attr-dict `:` type($sourceV1)

This operation interleaves elements from two input SVE vectors, returning two new SVE vectors (resultV1 and resultV2), which contain the low and high halves of the result respectively.

Example:

// sourceV1 = [ A1, A2, A3, ... An ]
// sourceV2 = [ B1, B2, B3, ... Bn ]
// (resultV1, resultV2) = [ A1, B1, A2, B2, A3, B3, ... An, Bn ]
%resultV1, %resultV2 = arm_sve.zip.x2 %sourceV1, %sourceV2 : vector<[16]xi8>

Note: This requires SME 2 (+sme2 in LLVM target features)

Source

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
sourceV1an SVE vector with element size <= 64-bit
sourceV2an SVE vector with element size <= 64-bit

Results: 

ResultDescription
resultV1an SVE vector with element size <= 64-bit
resultV2an SVE vector with element size <= 64-bit

arm_sve.zip.x4 (arm_sve::ZipX4Op) 

Multi-vector four-way zip op

Syntax:

operation ::= `arm_sve.zip.x4` $sourceV1 `,` $sourceV2 `,` $sourceV3 `,` $sourceV4 attr-dict
              `:` type($sourceV1)

This operation interleaves elements from four input SVE vectors, returning four new SVE vectors, each of which contain a quarter of the result. The first quarter will be in resultV1, second in resultV2, third in resultV3, and fourth in resultV4.

// sourceV1 = [ A1, A2, ... An ]
// sourceV2 = [ B1, B2, ... Bn ]
// sourceV3 = [ C1, C2, ... Cn ]
// sourceV4 = [ D1, D2, ... Dn ]
// (resultV1, resultV2, resultV3, resultV4)
//   = [ A1, B1, C1, D1, A2, B2, C2, D2, ... An, Bn, Cn, Dn ]
%resultV1, %resultV2, %resultV3, %resultV4 = arm_sve.zip.x4
  %sourceV1, %sourceV2, %sourceV3, %sourceV4 : vector<[16]xi8>

Warning: The result of this op is undefined for 64-bit elements on hardware with less than 256-bit vectors!

Note: This requires SME 2 (+sme2 in LLVM target features)

Source

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands: 

OperandDescription
sourceV1an SVE vector with element size <= 64-bit
sourceV2an SVE vector with element size <= 64-bit
sourceV3an SVE vector with element size <= 64-bit
sourceV4an SVE vector with element size <= 64-bit

Results: 

ResultDescription
resultV1an SVE vector with element size <= 64-bit
resultV2an SVE vector with element size <= 64-bit
resultV3an SVE vector with element size <= 64-bit
resultV4an SVE vector with element size <= 64-bit