|
| std::optional< vector::CombiningKind > | mlir::linalg::getCombinerOpKind (Operation *combinerOp) |
| | Return vector::CombiningKind for the given op. More...
|
| |
| Value | mlir::linalg::bufferizeToAllocation (RewriterBase &rewriter, const BufferizeToAllocationOptions &options, tensor::PadOp padOp, Attribute memorySpace={}, Operation *insertionPoint=nullptr) |
| | Materialize a buffer allocation for the given tensor.pad op and lower the op to linalg.fill/linalg.generic + bufferization.materialize_in_destination. More...
|
| |
| Value | mlir::linalg::bufferizeToAllocation (RewriterBase &rewriter, const BufferizeToAllocationOptions &options, vector::MaskOp maskOp, Attribute memorySpace={}, Operation *insertionPoint=nullptr) |
| | Materialize a buffer allocation for the given vector.mask op and bufferize the op, including its region. More...
|
| |
| Value | mlir::linalg::bufferizeToAllocation (RewriterBase &rewriter, const BufferizeToAllocationOptions &options, bufferization::AllocTensorOp allocTensorOp, Attribute memorySpace={}, Operation *insertionPoint=nullptr) |
| | Materialize a buffer allocation for the given bufferization.alloc_tensor op and lower the op to memref.alloc + memref.tensor_store. More...
|
| |
| Value | mlir::linalg::bufferizeToAllocation (RewriterBase &rewriter, const BufferizeToAllocationOptions &options, Operation *op, Attribute memorySpace={}, Operation *insertionPoint=nullptr) |
| | Bufferize the given op with tensor semantics and materialize the result in a newly allocated buffer. More...
|
| |
| LogicalResult | mlir::linalg::linalgOpAnchoredEmptyTensorEliminationStep (RewriterBase &rewriter, Operation *op, bufferization::OneShotAnalysisState &state) |
| | Try to eliminate tensor::EmptyOps inside op that are anchored on a LinalgOp. More...
|
| |
| bool | mlir::linalg::areElementwiseOpsFusable (OpOperand *fusedOperand) |
| | Return true if two linalg.generic operations with producer/consumer relationship through fusedOperand can be fused using elementwise op fusion. More...
|
| |
| LogicalResult | mlir::linalg::promoteSubviewsPrecondition (Operation *op, LinalgPromotionOptions options) |
| | Promote memref.subviews feeding linalg-on-buffers operations. More...
|
| |
| LogicalResult | mlir::linalg::vectorizeOpPrecondition (Operation *op, ArrayRef< int64_t > inputVectorSizes={}, ArrayRef< bool > inputScalableVecDims={}, bool vectorizeNDExtract=false, bool flatten1DDepthwiseConv=false) |
| | Return success if the operation can be vectorized. More...
|
| |
| LogicalResult | mlir::linalg::dropUnitDims (RewriterBase &rewriter, GenericOp genericOp, const ControlDropUnitDims &options) |
| |
| FailureOr< ElementwiseOpFusionResult > | mlir::linalg::fuseElementwiseOps (RewriterBase &rewriter, OpOperand *fusedOperand) |
| |
| SmallVector< Value > | mlir::linalg::peelLoop (RewriterBase &rewriter, Operation *op) |
| | Try to peel and canonicalize loop op and return the new result. More...
|
| |
| void | mlir::linalg::peelLoops (RewriterBase &rewriter, ArrayRef< scf::ForOp > loops) |
| | Peel 'loops' and applies affine_min/max bounds simplification on the fly where relevant. More...
|
| |
| LogicalResult | mlir::linalg::rewriteAsPaddedOp (RewriterBase &rewriter, LinalgOp opToPad, const LinalgPaddingOptions &options, LinalgOp &paddedOp, SmallVector< Value > &replacements, SmallVector< tensor::PadOp > &padOps) |
| | Pad the iterator dimensions paddingDimensions of all opToPad operands to a static bounding box. More...
|
| |
| FailureOr< PackingResult > | mlir::linalg::detail::buildPackingLoopNest (RewriterBase &rewriter, tensor::PadOp opToHoist, scf::ForOp outermostEnclosingForOp, ArrayRef< int64_t > transposeVector) |
| | Build the packing loop nest required to hoist opToHoist above outermostEnclosingForOp. More...
|
| |
| FailureOr< Value > | mlir::linalg::hoistPaddingOnTensors (RewriterBase &rewriter, tensor::PadOp opToHoist, int64_t numLoops, ArrayRef< int64_t > transposeVector, tensor::PadOp &hoistedOp, SmallVectorImpl< GenericOp > &transposeOps) |
| | Mechanically hoist padding operations on tensors by numLoops into a new, generally larger tensor. More...
|
| |
| FailureOr< Value > | mlir::linalg::hoistPaddingOnTensors (tensor::PadOp opToHoist, int64_t numLoops, ArrayRef< int64_t > transposeVector, tensor::PadOp &hoistedOp, SmallVectorImpl< GenericOp > &transposeOps) |
| | Calls into hoistPaddingOnTensors with a local IRRewriter. More...
|
| |
| FailureOr< LinalgOp > | mlir::linalg::padAndHoistLinalgOp (RewriterBase &rewriter, LinalgOp linalgOp, const LinalgPaddingOptions &options) |
| | Apply padding and hoisting to linalgOp according to the configuration specified in options. More...
|
| |
| std::pair< TilingInterface, TilingInterface > | mlir::linalg::splitOp (RewriterBase &rewriter, TilingInterface op, unsigned dimension, OpFoldResult splitPoint) |
| | Split the given op into two parts along the given iteration space dimension at the specified splitPoint, and return the two parts. More...
|
| |
| FailureOr< TiledLinalgOp > | mlir::linalg::tileLinalgOp (RewriterBase &b, LinalgOp op, const LinalgTilingOptions &options) |
| |
| FailureOr< GenericOp > | mlir::linalg::interchangeGenericOp (RewriterBase &rewriter, GenericOp genericOp, ArrayRef< unsigned > interchangeVector) |
| | Interchange the iterator_types and iterator_maps dimensions and adapts the index accesses of op. More...
|
| |
| FailureOr< GenericOp > | mlir::linalg::generalizeNamedOp (RewriterBase &rewriter, LinalgOp namedOp) |
| | Create a GenericOp from the given named operation namedOp and replace namedOp. More...
|
| |
| FailureOr< LinalgOp > | mlir::linalg::specializeGenericOp (RewriterBase &rewriter, GenericOp genericOp) |
| | Create a namedOp from the given GenericOp and replace the GenericOp. More...
|
| |
| FailureOr< PromotionInfo > | mlir::linalg::promoteSubviewAsNewBuffer (OpBuilder &b, Location loc, memref::SubViewOp subView, const AllocBufferCallbackFn &allocationFn, DataLayout &layout) |
| |
| FailureOr< LinalgOp > | mlir::linalg::promoteSubViews (OpBuilder &b, LinalgOp op, const LinalgPromotionOptions &options) |
| | Promote the subViews into a new buffer allocated at the insertion point b. More...
|
| |
| std::optional< Value > | mlir::linalg::allocateWorkgroupMemory (OpBuilder &builder, memref::SubViewOp subview, ArrayRef< Value > sizeBounds, DataLayout &) |
| | Allocate the subview in the GPU workgroup memory. More...
|
| |
| LogicalResult | mlir::linalg::deallocateWorkgroupMemory (OpBuilder &, Value) |
| | In case of GPU group memory there is no need to deallocate. More...
|
| |
| LogicalResult | mlir::linalg::copyToWorkgroupMemory (OpBuilder &b, Value src, Value dst) |
| | Create Memref copy operations and add gpu barrier guards before and after the copy operation to ensure data integrity. More...
|
| |
| std::optional< Value > | mlir::linalg::allocateGPUPrivateMemory (OpBuilder &builder, memref::SubViewOp subview, ArrayRef< Value > sizeBounds, DataLayout &) |
| | Allocate the subview in the GPU private memory. More...
|
| |
| LogicalResult | mlir::linalg::copyToGPUPrivateMemory (OpBuilder &b, Value src, Value dst) |
| | Normal copy to between src and dst. More...
|
| |
| LogicalResult | mlir::linalg::deallocateGPUPrivateMemory (OpBuilder &, Value) |
| | In case of GPU private memory there is no need to deallocate since the memory is freed when going outside of the scope. More...
|
| |
| LogicalResult | mlir::linalg::vectorize (RewriterBase &rewriter, Operation *op, ArrayRef< int64_t > inputVectorSizes={}, ArrayRef< bool > inputScalableVecDims={}, bool vectorizeNDExtract=false, bool flatten1DDepthwiseConv=false) |
| | Emit a suitable vector form for an operation. More...
|
| |
| LogicalResult | mlir::linalg::vectorizeCopy (RewriterBase &builder, memref::CopyOp copyOp) |
| | Emit a suitable vector form for a Copy op with fully static shape. More...
|
| |
| FailureOr< LinalgLoops > | mlir::linalg::linalgOpToLoops (RewriterBase &rewriter, LinalgOp linalgOp) |
| | Emit a loop nest of scf.for with the proper body for linalgOp. More...
|
| |
| FailureOr< LinalgLoops > | mlir::linalg::linalgOpToParallelLoops (RewriterBase &rewriter, LinalgOp linalgOp) |
| | Emit a loop nest of scf.parallel with the proper body for linalgOp. More...
|
| |
| FailureOr< LinalgLoops > | mlir::linalg::linalgOpToAffineLoops (RewriterBase &rewriter, LinalgOp linalgOp) |
| | Emit a loop nest of affine.for with the proper body for linalgOp. More...
|
| |
| std::tuple< SmallVector< Range, 4 >, LoopIndexToRangeIndexMap > | mlir::linalg::makeTiledLoopRanges (RewriterBase &b, Location loc, AffineMap map, ArrayRef< OpFoldResult > allShapeSizes, ArrayRef< OpFoldResult > allTileSizes) |
| |
| FailureOr< MultiSizeSpecification > | mlir::linalg::computeMultiTileSizes (OpBuilder &builder, LinalgOp op, unsigned dimension, OpFoldResult targetSize, OpFoldResult divisor, bool emitAssertions=true) |
| | Emits the IR computing the multi-sized tiling specification with two tile sizes not exceeding targetSize, each divisible by sizeDivisor, such that there exist numbers of tiles with these sizes that fully cover the given iteration space dimension of the structured op. More...
|
| |
| FailureOr< StaticMultiSizeSpecification > | mlir::linalg::computeStaticMultiTileSizes (LinalgOp op, unsigned dimension, int64_t targetSize, int64_t divisor) |
| |
| FailureOr< ForallTilingResult > | mlir::linalg::tileToForallOp (RewriterBase &builder, TilingInterface op, ArrayRef< OpFoldResult > numThreads, std::optional< ArrayAttr > mapping) |
| |
| FailureOr< ForallTilingResult > | mlir::linalg::tileToForallOpUsingTileSizes (RewriterBase &builder, TilingInterface op, ArrayRef< OpFoldResult > tileSizes, std::optional< ArrayAttr > mapping) |
| | Same as tileToForallOp, but calculate the number of threads required using the given tileSizes. More...
|
| |
| FailureOr< ForallReductionTilingResult > | mlir::linalg::tileReductionUsingForall (RewriterBase &b, PartialReductionOpInterface op, ArrayRef< OpFoldResult > numThreads, ArrayRef< OpFoldResult > tileSizes={}, std::optional< ArrayAttr > mapping=std::nullopt) |
| | Method to tile a reduction to parallel iterations computing partial reductions. More...
|
| |
| void | mlir::linalg::transformIndexOps (RewriterBase &b, LinalgOp op, SmallVectorImpl< Value > &ivs, const LoopIndexToRangeIndexMap &loopIndexToRangeIndex) |
| | All indices returned by IndexOp should be invariant with respect to tiling. More...
|
| |
| FailureOr< SplitReductionResult > | mlir::linalg::splitReduction (RewriterBase &b, LinalgOp op, const ControlSplitReductionFn &controlSplitReductionFn, bool useAlloc=false) |
| |
| FailureOr< SplitReductionResult > | mlir::linalg::splitReductionByScaling (RewriterBase &b, LinalgOp op, const ControlSplitReductionFn &controlSplitReductionFn, bool useAlloc=false) |
| | Scaling-based implementation of the split reduction transformation. More...
|
| |
| bool | mlir::linalg::isDimSequencePreserved (AffineMap map, ReassociationIndicesRef dimSequence) |
| | Return true if a given sequence of dimensions are contiguous in the range of the specified indexing map. More...
|
| |
| bool | mlir::linalg::areDimSequencesPreserved (ArrayRef< AffineMap > maps, ArrayRef< ReassociationIndices > dimSequences) |
| | Return true if all sequences of dimensions specified in dimSequences are contiguous in all the ranges of the maps. More...
|
| |
| FailureOr< CollapseResult > | mlir::linalg::collapseOpIterationDims (LinalgOp op, ArrayRef< ReassociationIndices > foldedIterationDims, RewriterBase &rewriter) |
| | Collapses dimensions of linalg.generic/linalg.copy operation. More...
|
| |
| FailureOr< LowerPackResult > | mlir::linalg::lowerPack (RewriterBase &rewriter, tensor::PackOp packOp) |
| | Rewrite pack as pad + reshape + transpose. More...
|
| |
| FailureOr< LowerUnPackOpResult > | mlir::linalg::lowerUnPack (RewriterBase &rewriter, tensor::UnPackOp unPackOp) |
| | Rewrite pack as empty + transpose + reshape + extract_slice. More...
|
| |
| FailureOr< PackResult > | mlir::linalg::pack (RewriterBase &rewriter, linalg::LinalgOp linalgOp, ArrayRef< OpFoldResult > packedSizes) |
| | Implement packing of a single LinalgOp by packedSizes. More...
|
| |
| FailureOr< PackTransposeResult > | mlir::linalg::packTranspose (RewriterBase &rewriter, tensor::PackOp packOp, linalg::LinalgOp linalgOp, tensor::UnPackOp maybeUnPackOp, ArrayRef< int64_t > outerPerm, ArrayRef< int64_t > innerPerm) |
| | Transpose a single PackOp -> LinalgOp -> UnPackOp chain and return the transposed PackOp -> LinalgOp -> UnPackOp chain after replacements. More...
|
| |
| FailureOr< PackResult > | mlir::linalg::packMatmulGreedily (RewriterBase &rewriter, LinalgOp linalgOp, ArrayRef< OpFoldResult > mnkPackedSizes, ArrayRef< int64_t > mnkPaddedSizesNextMultipleOf, ArrayRef< int64_t > mnkOrder) |
| | Pack a LinalgOp by greedily inferring matmul dimensions (m, n, k) where m and n are proper parallel dimensions and k is a proper reduction dimension. More...
|
| |
| FailureOr< Operation * > | mlir::linalg::rewriteInDestinationPassingStyle (RewriterBase &rewriter, tensor::FromElementsOp fromElementsOp) |
| | Rewrite tensor.from_elements to linalg.generic. More...
|
| |
| FailureOr< Operation * > | mlir::linalg::rewriteInDestinationPassingStyle (RewriterBase &rewriter, tensor::GenerateOp generateOp) |
| | Rewrite tensor.generate to linalg.generic. More...
|
| |
| FailureOr< Operation * > | mlir::linalg::rewriteInDestinationPassingStyle (RewriterBase &rewriter, tensor::PadOp padOp) |
| | Rewrite tensor.pad to linalg.generic + tensor.insert_slice. More...
|
| |
| FailureOr< std::pair< Operation *, Operation * > > | mlir::linalg::rewriteInIm2Col (RewriterBase &rewriter, linalg::Conv2DNhwcHwcfOp convOp) |
| | Convert linalg.conv_2d_nhwc_hwcf into linalg.generic (for img2col packing) and linalg.matmul. More...
|
| |
| FailureOr< std::pair< Operation *, Operation * > > | mlir::linalg::rewriteInIm2Col (RewriterBase &rewriter, linalg::Conv2DNhwcFhwcOp convOp) |
| | Same as the above but for Fhwc channel orderings in the filter. More...
|
| |
| FailureOr< std::pair< Operation *, Operation * > > | mlir::linalg::rewriteInIm2Col (RewriterBase &rewriter, linalg::DepthwiseConv2DNhwcHwcOp convOp) |
| | Similar to rewriteInIm2Col with linalg::Conv2DNhwcHwcfOp except there is no reduction among the input channels so each convolution can be a matrix-vector product and by transposing both input filter so channels are outer most the computation is a batched matrix-vector product. More...
|
| |
| FailureOr< std::pair< Operation *, Operation * > > | mlir::linalg::rewriteInIm2Col (RewriterBase &rewriter, linalg::Conv2DNchwFchwOp convOp) |
| | Similar to rewriteInIm2Col with linalg::Conv2DNhwcHwcfOp except because the channels are to the left of the image shape dimensions, the position of the contraction dimension in the resulting matmul is reversed. More...
|
| |
| FailureOr< Operation * > | mlir::linalg::transposeConv2D (RewriterBase &rewriter, linalg::Conv2DNhwcFhwcOp op) |
| | Convert linalg.conv_2d_nhwc_fhwc(_q) to linalg.conv_2d_nhwc_hwcf(_q) by materializing transpose. More...
|
| |
| FailureOr< Operation * > | mlir::linalg::transposeConv2D (RewriterBase &rewriter, linalg::Conv2DNhwcFhwcQOp op) |
| |
| RewritePatternSet | mlir::linalg::getLinalgTilingCanonicalizationPatterns (MLIRContext *ctx) |
| | Canonicalization patterns relevant to apply after tiling patterns. More...
|
| |
| void | mlir::linalg::populateLinalgTilingCanonicalizationPatterns (RewritePatternSet &patterns) |
| |
| void | mlir::linalg::populateLinalgNamedOpsGeneralizationPatterns (RewritePatternSet &patterns) |
| | Linalg generalization patterns. More...
|
| |
| void | mlir::linalg::populateDecomposeConvolutionPatterns (RewritePatternSet &patterns, PatternBenefit benefit=1) |
| | Linalg decompose convolutions patterns. More...
|
| |
| void | mlir::linalg::populateConvertConv2DToImg2ColPatterns (RewritePatternSet &patterns) |
| | Populates patterns to transform linalg.conv_2d_xxx operations into linalg.generic (for img2col packing) and linalg.matmul. More...
|
| |
| void | mlir::linalg::populatePadOpVectorizationPatterns (RewritePatternSet &patterns, PatternBenefit baseBenefit=1) |
| | Populates patterns with patterns that vectorize tensor.pad. More...
|
| |
| void | mlir::linalg::populateDecomposeLinalgOpsPattern (RewritePatternSet &patterns, bool removeDeadArgsAndResults=true) |
| | Populate patterns for splitting a LinalgOp with multiple statements within its payload into multiple GenericOp that have a single statement. More...
|
| |
| void | mlir::linalg::populateConvertToDestinationStylePatterns (RewritePatternSet &patterns) |
| | Populate patterns that convert non-destination-style ops to destination style ops. More...
|
| |
| void | mlir::linalg::populateConvolutionVectorizationPatterns (RewritePatternSet &patterns, PatternBenefit benefit=1) |
| | Populate patterns for vectorizing low-D convolution ops. More...
|
| |
| void | mlir::linalg::populateElementwiseToLinalgConversionPatterns (RewritePatternSet &patterns) |
| | Populate patterns that convert ElementwiseMappable ops to linalg parallel loops. More...
|
| |
| void | mlir::linalg::populateSparseTensorRewriting (RewritePatternSet &patterns) |
| | Populate patterns that are only useful in the context of sparse tensors. More...
|
| |
| void | mlir::linalg::populateElementwiseOpsFusionPatterns (RewritePatternSet &patterns, const ControlFusionFn &controlElementwiseOpFusion) |
| | Patterns for fusing linalg operation on tensors. More...
|
| |
| void | mlir::linalg::populateDataLayoutPropagationPatterns (RewritePatternSet &patterns, const ControlPropagationFn &controlPackUnPackPropagation) |
| | Patterns to bubble up or down data layout ops across other operations. More...
|
| |
| void | mlir::linalg::populateEraseUnusedOperandsAndResultsPatterns (RewritePatternSet &patterns) |
| | Pattern to remove dead operands and results of linalg.generic operations. More...
|
| |
| void | mlir::linalg::populateEraseUnnecessaryInputsPatterns (RewritePatternSet &patterns) |
| | Patterns to promote inputs to outputs and remove unused inputs of linalg.generic ops. More...
|
| |
| void | mlir::linalg::populateCollapseDimensions (RewritePatternSet &patterns, const GetCollapsableDimensionsFn &controlCollapseDimensions) |
| | Pattern to collapse dimensions in a linalg.generic op. More...
|
| |
| void | mlir::linalg::populateFoldReshapeOpsByExpansionPatterns (RewritePatternSet &patterns, const ControlFusionFn &controlFoldingReshapes) |
| | Patterns to fold an expanding (collapsing) tensor_reshape operation with its producer (consumer) generic operation by expanding the dimensionality of the loop in the generic op. More...
|
| |
| void | mlir::linalg::populateFoldReshapeOpsByCollapsingPatterns (RewritePatternSet &patterns, const ControlFusionFn &controlFoldingReshapes) |
| | Patterns to fold an expanding tensor.expand_shape operation with its producer generic operation by collapsing the dimensions of the generic op. More...
|
| |
| void | mlir::linalg::populateConstantFoldLinalgOperations (RewritePatternSet &patterns, const ControlFusionFn &controlFn) |
| | Patterns to constant fold Linalg operations. More...
|
| |
| void | mlir::linalg::populateFuseTensorPadWithProducerLinalgOpPatterns (RewritePatternSet &patterns) |
| | Pattern to fuse a tensor.pad operation with the producer of its source, if the producer is a linalg operation with all parallel iterator types. More...
|
| |
| void | mlir::linalg::populateLinalgNamedOpConversionPatterns (RewritePatternSet &patterns) |
| | Patterns to convert from one named op to another. More...
|
| |
| void | mlir::linalg::populateFoldUnitExtentDimsPatterns (RewritePatternSet &patterns, ControlDropUnitDims &options) |
| | Patterns to fold unit-extent dimensions in operands/results of linalg ops on tensors via reassociative reshape ops. More...
|
| |
| void | mlir::linalg::populateMoveInitOperandsToInputPattern (RewritePatternSet &patterns) |
| | A pattern that converts init operands to input operands. More...
|
| |
| void | mlir::linalg::populateInlineConstantOperandsPatterns (RewritePatternSet &patterns) |
| | Patterns that are used to inline constant operands into linalg generic ops. More...
|
| |
| void | mlir::linalg::populateBubbleUpExtractSliceOpPatterns (RewritePatternSet &patterns) |
| | Patterns that are used to bubble up extract slice op above linalg op. More...
|
| |
| void | mlir::linalg::populateSwapExtractSliceWithFillPatterns (RewritePatternSet &patterns) |
| | Adds patterns that waps tensor.extract_slice(linalg.fill(cst, init)) into linalg.fill(cst, tensor.extract_slice(init)). More...
|
| |
| void | mlir::linalg::populateSplitReductionPattern (RewritePatternSet &patterns, const ControlSplitReductionFn &controlSplitReductionFn, bool useAlloc=false) |
| | Patterns to apply splitReduction below. More...
|
| |
