|
MLIR 23.0.0git
|
Namespaces | |
| namespace | uArch |
Classes | |
| struct | UnrollOptions |
| Options to control the XeGPU unrolling. More... | |
Functions | |
| void | registerTransformDialectExtension (DialectRegistry ®istry) |
| void | populateXeGPUFoldAliasOpsPatterns (RewritePatternSet &patterns) |
| Appends patterns for folding aliasing ops into XeGPU ops into patterns. | |
| void | populateXeGPUPeepHoleOptimizerPatterns (RewritePatternSet &patterns) |
| Appends patterns for optimizing block load operations into patterns. | |
| void | populateXeGPUSubgroupDistributePatterns (RewritePatternSet &patterns) |
| Appends patterns for XeGPU SIMT distribution into patterns. | |
| void | populateXeGPUMoveFuncBodyToWarpOpPatterns (RewritePatternSet &patterns) |
| Appends patterns for moving function body into gpu.warp_execute_on_lane0 op. | |
| void | populateXeGPUWgToSgDistributePatterns (RewritePatternSet &patterns) |
| Appends patterns for XeGPU workgroup to subgroup distribution into patterns. | |
| void | populateXeGPUUnrollPatterns (RewritePatternSet &patterns, const UnrollOptions &options) |
| Collect a set of patterns to unroll xegpu operations to a smaller shapes. | |
| SmallVector< Value > | flattenValues (ArrayRef< ValueRange > values) |
| Flatten a set of ValueRange into a single SmallVector<Value> | |
| FailureOr< VectorType > | getDistributedVectorType (xegpu::TensorDescType tdescTy) |
| If tensor descriptor has a layout attribute it is used in SIMT mode. | |
| FailureOr< VectorType > | getDistributedVectorType (VectorType originalType, LayoutAttr layout) |
| Helper to get the distributed vector type for a given vector type according to a given LayoutAttr. | |
| SmallVector< Value > | extractVectorsWithShapeFromValue (OpBuilder &builder, Location loc, Value value, ArrayRef< int64_t > shape) |
| Extract a set of small vectors from a value with a given shape using vector.extract_stride_slice. | |
| Value | createVectorWithShapeFromValues (OpBuilder &builder, Location loc, ValueRange values, ArrayRef< int64_t > shape) |
| Create a vector of shape from a set of values using vector.insert_stride_slice. | |
| void | doSCFStructuralTypeConversionWithTensorType (Operation *op, TypeConverter converter) |
| Do type conversion for SCF structural ops, e.g., scf.for using SCF structure type convertion patterns. | |
| std::optional< std::string > | getChipStr (Operation *op) |
| Retrieves the chip string from the XeVM target attribute of the parent GPU module operation. | |
| SmallVector< OpFoldResult > | addElementwise (OpBuilder &builder, Location loc, ArrayRef< OpFoldResult > lhs, ArrayRef< OpFoldResult > rhs) |
| Generates element-wise addition ops of two arrays with same length. | |
| SmallVector< OpFoldResult > | addWithRightAligned (OpBuilder &builder, Location loc, ArrayRef< OpFoldResult > lhs, ArrayRef< OpFoldResult > rhs) |
| Generates element-wise addition ops of two arrays with automatic alignment. | |
| template<typename T> | |
| int | getLargestDivisor (T dim, ArrayRef< T > candidates, ArrayRef< T > candidateMultiples={}) |
| Helper Function to find a proper instruction multiple for the user-supplied sg-level data shape (diven by dim). | |
| std::string | getTemporaryLayoutName (const OpOperand &operand) |
| Return the attribute name for the OpOperand to attach DistributeLayoutAttr. | |
| std::string | getTemporaryLayoutName (const OpResult result) |
| Return the attribute name for the OpResult to attach DistributeLayoutAttr. | |
| DistributeLayoutAttr | getDistributeLayoutAttr (const Value value) |
| Retrieves the DistributeLayoutAttr associated with a given Value. | |
| DistributeLayoutAttr | getDistributeLayoutAttr (const OpOperand &opr) |
| Retrieves the DistributeLayoutAttr associated with a given OpOperand. | |
| template<typename T, typename = std::enable_if_t<std::is_same_v<T, OpOperand> || std::is_same_v<T, OpResult>>> | |
| void | removeLayoutAttr (const T &operandOrResult) |
| Removes the LayoutAttr for a given OpOperand or OpResult if it exists. | |
| void | removeLayoutAttrs (Operation *op) |
| Removes the DistributeLayoutAttr for each OpOperand and OpResult of the given operation if they exist. | |
| SmallVector< NamedAttribute > | dropSgLayoutAndDataOnAttrs (ArrayRef< NamedAttribute > attrs) |
| Updates the NamedAttribute sequence by dropping sg-layout and sg-data information from any DistributeLayoutAttr found. | |
| SmallVector< NamedAttribute > | dropInstDataOnAttrs (ArrayRef< NamedAttribute > attrs) |
| Updates the NamedAttribute sequence by dropping inst-data information from any DistributeLayoutAttr found. | |
| void | setDistributeLayoutAttr (const OpResult &Result, const DistributeLayoutAttr layout) |
| [to-be-deprecated] Sets the DistributeLayoutAttr for a given OpResult user should use setAnchorLayout instead | |
| void | setDistributeLayoutAttr (const OpOperand &opr, const DistributeLayoutAttr layout) |
| [to-be-deprecated] Sets the DistributeLayoutAttr for a given OpOperand user should use setAnchorLayout instead | |
| template<typename T, typename = std::enable_if_t<std::is_same_v<T, OpOperand> || std::is_same_v<T, OpResult>>> | |
| DistributeLayoutAttr | getTemporaryLayout (const T &operandOrResult) |
| get and set distribute layout attribute for non-anchor operations (and offsets/masks of load/store ops before we get rid of their temp attrs) | |
| template<typename T, typename = std::enable_if_t<std::is_same_v<T, OpOperand> || std::is_same_v<T, OpResult>>> | |
| void | setTemporaryLayout (const T &operandOrResult, const DistributeLayoutAttr layout) |
| void | recoverTemporaryLayoutsDeprecated (Operation *op) |
| [to-be-deprecated] Set the DistributeLayoutAttr for each OpOperand and OpResult of of the given operation. | |
| bool | recoverTemporaryLayouts (Operation *rootOp) |
| Attach layout attributes to all vector-type operands of operations within the given operation's region. | |
| static SmallVector< SmallVector< Value > > | genCoordinates (OpBuilder &builder, Location loc, SmallVector< Value > delinearizedId, ArrayRef< int64_t > subShapesLayout, ArrayRef< int64_t > subShape, ArrayRef< int64_t > srcShape) |
| static SetVector< int64_t > | mapSlicedDimsToParentSpace (const SetVector< int64_t > &dimsToMap, ArrayRef< int64_t > sliceDims) |
| template<typename ArithOp> | |
| OpFoldResult | genBinOp (OpFoldResult a, OpFoldResult b, Location loc, OpBuilder &builder) |
| SmallVector< OpFoldResult > | getBlockedOffsets (OpBuilder &builder, Location loc, ArrayRef< OpFoldResult > offsets, ArrayRef< int64_t > blockShape) |
| SmallVector< OpFoldResult > mlir::xegpu::addElementwise | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ArrayRef< OpFoldResult > | lhs, | ||
| ArrayRef< OpFoldResult > | rhs ) |
Generates element-wise addition ops of two arrays with same length.
Definition at line 672 of file XeGPUUtils.cpp.
References mlir::OpBuilder::createOrFold(), mlir::getValueOrCreateConstantIndexOp(), lhs, and rhs.
Referenced by addWithRightAligned().
| SmallVector< OpFoldResult > mlir::xegpu::addWithRightAligned | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ArrayRef< OpFoldResult > | lhs, | ||
| ArrayRef< OpFoldResult > | rhs ) |
Generates element-wise addition ops of two arrays with automatic alignment.
When the input arrays have different sizes, the shorter array is right-aligned with the longer array, and the unmatched leading elements from the longer array are preserved unchanged. This is commonly used for offset computation where higher-dimensional offsets need to be added to lower-dimensional adjustments.
Example: lhs = [l1, l2, l3], rhs = [r1, r2] Result: [11, l2+r1, l3+r2]
Definition at line 697 of file XeGPUUtils.cpp.
References addElementwise(), b, lhs, and rhs.
| Value mlir::xegpu::createVectorWithShapeFromValues | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ValueRange | values, | ||
| ArrayRef< int64_t > | shape ) |
Create a vector of shape from a set of values using vector.insert_stride_slice.
Definition at line 481 of file XeGPUUtils.cpp.
References mlir::DenseElementsAttr::get(), mlir::getType(), mlir::ValueRange::getTypes(), mlir::Builder::getZeroAttr(), and result.
| void mlir::xegpu::doSCFStructuralTypeConversionWithTensorType | ( | Operation * | op, |
| TypeConverter | converter ) |
Do type conversion for SCF structural ops, e.g., scf.for using SCF structure type convertion patterns.
Since VectorType cannot carry the layout attribute, which is needed to guide the type conversion for XeGPU, they are first converted into RankedTensorType, where the layout attribute can be attached. And then upstream SCF structural type conversion patterns are applied with the provided converter. TODO: This is a temporary solution. We should refactor it when context-aware type conversion is available.
Definition at line 506 of file XeGPUUtils.cpp.
References mlir::WalkResult::advance(), flattenValues(), mlir::Operation::getContext(), getDistributeLayoutAttr(), mlir::Operation::getOperandTypes(), mlir::Operation::getOpResults(), mlir::Operation::getParentOp(), mlir::Operation::getResultTypes(), mlir::Value::getType(), mlir::patterns, mlir::scf::populateSCFStructuralTypeConversionsAndLegality(), result, mlir::Value::setType(), mlir::WalkResult::skip(), success(), target, and mlir::Operation::walk().
| SmallVector< NamedAttribute > mlir::xegpu::dropInstDataOnAttrs | ( | ArrayRef< NamedAttribute > | attrs | ) |
Updates the NamedAttribute sequence by dropping inst-data information from any DistributeLayoutAttr found.
Definition at line 401 of file XeGPUUtils.cpp.
| SmallVector< NamedAttribute > mlir::xegpu::dropSgLayoutAndDataOnAttrs | ( | ArrayRef< NamedAttribute > | attrs | ) |
Updates the NamedAttribute sequence by dropping sg-layout and sg-data information from any DistributeLayoutAttr found.
Definition at line 383 of file XeGPUUtils.cpp.
| SmallVector< Value > mlir::xegpu::extractVectorsWithShapeFromValue | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | value, | ||
| ArrayRef< int64_t > | shape ) |
Extract a set of small vectors from a value with a given shape using vector.extract_stride_slice.
Definition at line 444 of file XeGPUUtils.cpp.
References mlir::computeShapeRatio(), mlir::Value::getType(), and result.
| SmallVector< Value > mlir::xegpu::flattenValues | ( | ArrayRef< ValueRange > | values | ) |
Flatten a set of ValueRange into a single SmallVector<Value>
convert ArrayRef<ValueRange> into SmallVector<Value>
Definition at line 31 of file XeGPUUtils.cpp.
References result.
Referenced by doSCFStructuralTypeConversionWithTensorType().
| OpFoldResult mlir::xegpu::genBinOp | ( | OpFoldResult | a, |
| OpFoldResult | b, | ||
| Location | loc, | ||
| OpBuilder & | builder ) |
Definition at line 991 of file XeGPUDialect.cpp.
References b, and mlir::getValueOrCreateConstantIndexOp().
|
static |
Definition at line 50 of file XeGPUDialect.cpp.
References mlir::computeElementwiseMul(), mlir::arith::ConstantIndexOp::create(), and mlir::OpBuilder::createOrFold().
| SmallVector< OpFoldResult > mlir::xegpu::getBlockedOffsets | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ArrayRef< OpFoldResult > | offsets, | ||
| ArrayRef< int64_t > | blockShape ) |
Definition at line 1016 of file XeGPUDialect.cpp.
| std::optional< std::string > mlir::xegpu::getChipStr | ( | Operation * | op | ) |
Retrieves the chip string from the XeVM target attribute of the parent GPU module operation.
Returns the chip identifier if found, or nullopt if no GPU module parent or XeVM target attribute exists.
Definition at line 653 of file XeGPUUtils.cpp.
References mlir::Operation::getParentOfType().
| FailureOr< VectorType > mlir::xegpu::getDistributedVectorType | ( | VectorType | originalType, |
| LayoutAttr | layout ) |
| FailureOr< VectorType > mlir::xegpu::getDistributedVectorType | ( | xegpu::TensorDescType | tdescTy | ) |
If tensor descriptor has a layout attribute it is used in SIMT mode.
In this mode, the distributed vector shape is determined as follows: Definitions: lane_data_size = lane_data[0] × lane_data[1] subgroup_size = lane_layout[0] × lane_layout[1] distribution_unit_size = subgroup_size × lane_data_size
Case 1: Regular loads/stores. The following conditions must be met:
Case 2: Block loads/stores Additional definitions: tensor_size = tensor_desc[0] * .. * tensor_desc[r-1] * array_length n_distribution_units = tensor_size / distribution_unit_size fragment_size = n_distribution_units * lane_data_size Given above definitions, the following conditions must be met:
Definition at line 39 of file XeGPUUtils.cpp.
| xegpu::DistributeLayoutAttr mlir::xegpu::getDistributeLayoutAttr | ( | const OpOperand & | opr | ) |
Retrieves the DistributeLayoutAttr associated with a given OpOperand.
It will first check the operand_layout_{id} of the owner operation. If not found, it will check the operand itself and its defining op.
Definition at line 152 of file XeGPUUtils.cpp.
References mlir::Operation::getAttrOfType(), mlir::detail::IROperandBase::getOwner(), getTemporaryLayoutName(), and mlir::Operation::hasAttr().
| xegpu::DistributeLayoutAttr mlir::xegpu::getDistributeLayoutAttr | ( | const Value | value | ) |
Retrieves the DistributeLayoutAttr associated with a given Value.
For TensorDescType values, the DistributeLayoutAttr is extracted from the TensorDescType itself. For other values, it is obtained from the attributes of the defining operation. Returns nullptr if no DistributeLayoutAttr is found.
Definition at line 115 of file XeGPUUtils.cpp.
References mlir::IROperand< DerivedT, IRValueT >::get(), mlir::Operation::getAttrOfType(), getDistributeLayoutAttr(), getTemporaryLayoutName(), mlir::Value::getType(), mlir::Operation::hasAttr(), and result.
Referenced by doSCFStructuralTypeConversionWithTensorType(), getDistributeLayoutAttr(), recoverTemporaryLayouts(), and recoverTemporaryLayoutsDeprecated().
| int mlir::xegpu::getLargestDivisor | ( | T | dim, |
| ArrayRef< T > | candidates, | ||
| ArrayRef< T > | candidateMultiples = {} ) |
Helper Function to find a proper instruction multiple for the user-supplied sg-level data shape (diven by dim).
candidates are uArch allowed shapes. candidateMultiples are uArch multiples of such shapes (i.e. block count or array length).
Definition at line 710 of file XeGPUUtils.cpp.
| DistributeLayoutAttr mlir::xegpu::getTemporaryLayout | ( | const T & | operandOrResult | ) |
get and set distribute layout attribute for non-anchor operations (and offsets/masks of load/store ops before we get rid of their temp attrs)
Referenced by xegpu::getTemporaryLayout< mlir::OpOperand >(), and xegpu::getTemporaryLayout< mlir::OpResult >().
| std::string mlir::xegpu::getTemporaryLayoutName | ( | const OpOperand & | operand | ) |
Return the attribute name for the OpOperand to attach DistributeLayoutAttr.
Definition at line 104 of file XeGPUUtils.cpp.
Referenced by getDistributeLayoutAttr(), getDistributeLayoutAttr(), removeLayoutAttr(), and setDistributeLayoutAttr().
| std::string mlir::xegpu::getTemporaryLayoutName | ( | const OpResult | result | ) |
Return the attribute name for the OpResult to attach DistributeLayoutAttr.
Definition at line 110 of file XeGPUUtils.cpp.
References result.
|
static |
Definition at line 627 of file XeGPUDialect.cpp.
| void mlir::xegpu::populateXeGPUFoldAliasOpsPatterns | ( | RewritePatternSet & | patterns | ) |
Appends patterns for folding aliasing ops into XeGPU ops into patterns.
Definition at line 63 of file XeGPUFoldAliasOps.cpp.
References mlir::patterns.
| void mlir::xegpu::populateXeGPUMoveFuncBodyToWarpOpPatterns | ( | RewritePatternSet & | patterns | ) |
Appends patterns for moving function body into gpu.warp_execute_on_lane0 op.
Definition at line 2049 of file XeGPUSubgroupDistribute.cpp.
References mlir::patterns.
| void mlir::xegpu::populateXeGPUPeepHoleOptimizerPatterns | ( | RewritePatternSet & | patterns | ) |
Appends patterns for optimizing block load operations into patterns.
Definition at line 505 of file XeGPUPeepHoleOptimizer.cpp.
References mlir::patterns.
| void mlir::xegpu::populateXeGPUSubgroupDistributePatterns | ( | RewritePatternSet & | patterns | ) |
Appends patterns for XeGPU SIMT distribution into patterns.
Definition at line 2029 of file XeGPUSubgroupDistribute.cpp.
References mlir::patterns.
| void mlir::xegpu::populateXeGPUUnrollPatterns | ( | RewritePatternSet & | patterns, |
| const UnrollOptions & | options ) |
Collect a set of patterns to unroll xegpu operations to a smaller shapes.
Users can control whether an operation to be unrolled or not, as well as its target shape via options structure. (via setting filterConstraint and nativeShape respectively, both of them are function refs taking op as input). An op is unrolled to the targetShape as follows, for each of its operands:
Definition at line 1036 of file XeGPUUnroll.cpp.
References options, and mlir::patterns.
| void mlir::xegpu::populateXeGPUWgToSgDistributePatterns | ( | RewritePatternSet & | patterns | ) |
Appends patterns for XeGPU workgroup to subgroup distribution into patterns.
Definition at line 1700 of file XeGPUWgToSgDistribute.cpp.
References mlir::patterns.
Attach layout attributes to all vector-type operands of operations within the given operation's region.
Reports an error if any vector operand lacks a layout attribute.
Definition at line 355 of file XeGPUUtils.cpp.
References mlir::WalkResult::advance(), mlir::Operation::emitError(), getDistributeLayoutAttr(), mlir::Operation::getName(), mlir::Operation::getOpOperands(), mlir::WalkResult::interrupt(), result, setDistributeLayoutAttr(), and mlir::Operation::walk().
[to-be-deprecated] Set the DistributeLayoutAttr for each OpOperand and OpResult of of the given operation.
If the operation contains regions, it is also applied recursively to the contained operations operation. TODO: To be replaced by recoverTemporaryLayouts()
Definition at line 338 of file XeGPUUtils.cpp.
References getDistributeLayoutAttr(), mlir::Operation::getOpOperands(), mlir::Operation::getOpResults(), result, setDistributeLayoutAttr(), and mlir::Operation::walk().
| void mlir::xegpu::registerTransformDialectExtension | ( | DialectRegistry & | registry | ) |
Definition at line 693 of file XeGPUTransformOps.cpp.
References mlir::DialectRegistry::addExtensions().
Referenced by mlir::registerAllExtensions().
| void mlir::xegpu::removeLayoutAttr | ( | const T & | operandOrResult | ) |
Removes the LayoutAttr for a given OpOperand or OpResult if it exists.
Definition at line 375 of file XeGPUUtils.cpp.
References getTemporaryLayoutName(), mlir::Operation::hasAttrOfType(), and mlir::Operation::removeAttr().
Referenced by removeLayoutAttrs(), xegpu::removeLayoutAttr< mlir::OpOperand >(), and xegpu::removeLayoutAttr< mlir::OpResult >().
Removes the DistributeLayoutAttr for each OpOperand and OpResult of the given operation if they exist.
If the operation contains regions, it is also applied recursively to the contained operations
Definition at line 426 of file XeGPUUtils.cpp.
References mlir::Operation::getOpOperands(), mlir::Operation::getOpResults(), mlir::Operation::hasAttrOfType(), mlir::Operation::removeAttr(), removeLayoutAttr(), result, and mlir::Operation::walk().
| void mlir::xegpu::setDistributeLayoutAttr | ( | const OpOperand & | opr, |
| const DistributeLayoutAttr | layout ) |
[to-be-deprecated] Sets the DistributeLayoutAttr for a given OpOperand user should use setAnchorLayout instead
Definition at line 252 of file XeGPUUtils.cpp.
References mlir::detail::IROperandBase::getOwner(), getTemporaryLayoutName(), mlir::Operation::hasAttrOfType(), and mlir::Operation::setAttr().
| void mlir::xegpu::setDistributeLayoutAttr | ( | const OpResult & | Result, |
| const DistributeLayoutAttr | layout ) |
[to-be-deprecated] Sets the DistributeLayoutAttr for a given OpResult user should use setAnchorLayout instead
Referenced by recoverTemporaryLayouts(), recoverTemporaryLayoutsDeprecated(), updateControlFlowOps(), and updateOp().
| void mlir::xegpu::setTemporaryLayout | ( | const T & | operandOrResult, |
| const DistributeLayoutAttr | layout ) |