|
MLIR
15.0.0git
|
Namespaces | |
| func_ext | |
Classes | |
| class | AlwaysCopyAnalysisState |
| This a "no analysis, always copy" AnalysisState. More... | |
| class | AnalysisState |
| AnalysisState provides a variety of helper functions for dealing with tensor values. More... | |
| class | BufferizationAliasInfo |
| The BufferizationAliasInfo class maintains a list of buffer aliases and equivalence classes to support bufferization. More... | |
| struct | BufferizationOptions |
| Options for BufferizableOpInterface-based bufferization. More... | |
| struct | BufferizationState |
| BufferizationState provides helper functions for performing bufferization rewrites and handling memref buffers. More... | |
| class | BufferizeTypeConverter |
| A helper type converter class that automatically populates the relevant materializations and type conversions for bufferization. More... | |
| class | BufferPlacementAllocs |
| A simple analysis that detects allocation operations. More... | |
| class | BufferPlacementTransformationBase |
| The base class for all BufferPlacement transformations. More... | |
| struct | DialectAnalysisState |
| Dialect-specific analysis state. More... | |
| class | OneShotAnalysisState |
| State for analysis-enabled bufferization. More... | |
| struct | OneShotBufferizationOptions |
| Options for analysis-enabled bufferization. More... | |
Typedefs | |
| using | AnchorMatchFn = std::function< bool(OpOperand &, SmallVector< Value > &)> |
| A function that matches anchor OpOperands for AllocTensorOp elimination. More... | |
| using | RewriteFn = std::function< Value(OpBuilder &, Location, OpOperand &)> |
| A function that rewrites matched anchors. More... | |
| using | PostAnalysisStepFn = std::function< LogicalResult(Operation *, AnalysisState &, BufferizationAliasInfo &, SmallVector< Operation * > &)> |
PostAnalysisStepFns can be registered with BufferizationOptions and are executed after the analysis, but before bufferization. More... | |
| using | PostAnalysisStepList = SmallVector< PostAnalysisStepFn > |
Enumerations | |
| enum | BufferRelation { BufferRelation::None, BufferRelation::Equivalent } |
| Specify fine-grain relationship between buffers to enable more analysis. More... | |
Functions | |
| bool | isFunctionArgument (Value value) |
Return true if the given value is a BlockArgument of a func::FuncOp. More... | |
| void | replaceOpWithBufferizedValues (RewriterBase &rewriter, Operation *op, ValueRange values) |
| Replace an op with replacement values. More... | |
| Value | lookupBuffer (RewriterBase &rewriter, Value tensor, const BufferizationOptions &options) |
| Lookup the buffer for the given value. More... | |
| template<typename OpTy , typename... Args> | |
| OpTy | replaceOpWithNewBufferizedOp (RewriterBase &rewriter, Operation *op, Args &&...args) |
| Replace an op with a new op. More... | |
| BaseMemRefType | getMemRefType (TensorType tensorType, const BufferizationOptions &options, MemRefLayoutAttrInterface layout={}, Attribute memorySpace={}) |
Return a MemRefType to which the tensorType can be bufferized. More... | |
| BaseMemRefType | getMemRefTypeWithFullyDynamicLayout (TensorType tensorType, Attribute memorySpace={}) |
| Return a MemRef type with fully dynamic layout. More... | |
| BaseMemRefType | getMemRefTypeWithStaticIdentityLayout (TensorType tensorType, Attribute memorySpace={}) |
| Return a MemRef type with a static identity layout (i.e., no layout map). More... | |
| LogicalResult | hoistBufferAllocations (Operation *op, const BufferizationOptions &options) |
| Try to hoist all new buffer allocations until the next hoisting barrier. More... | |
| LogicalResult | createAllocDeallocOps (Operation *op, const BufferizationOptions &options, bool onlyLeakingAllocs=false, bool *changed=nullptr) |
| Create alloc/dealloc ops as specified in the bufferization options. More... | |
| FailureOr< Value > | castOrReallocMemRefValue (OpBuilder &b, Value value, MemRefType type) |
| Try to cast the given ranked MemRef-typed value to the given ranked MemRef type. More... | |
| LogicalResult | foldToMemrefToTensorPair (RewriterBase &rewriter, ToMemrefOp toMemref, bool allowSameType=true) |
| Try to fold to_memref(to_tensor(x)). More... | |
| LogicalResult | eliminateAllocTensors (RewriterBase &rewriter, Operation *op, bufferization::AnalysisState &state, AnchorMatchFn anchorMatchFunc, RewriteFn rewriteFunc) |
Try to eliminate AllocTensorOps inside op. More... | |
| LogicalResult | insertSliceAnchoredAllocTensorEliminationStep (RewriterBase &rewriter, Operation *op, bufferization::AnalysisState &state) |
Try to eliminate AllocTensorOps inside op that are anchored on an InsertSliceOp, i.e., if it is eventually inserted into another tensor (and some other conditions are met). More... | |
| void | populateBufferizeMaterializationLegality (ConversionTarget &target) |
| Marks ops used by bufferization for type conversion materializations as "legal" in the given ConversionTarget. More... | |
| void | populateEliminateBufferizeMaterializationsPatterns (BufferizeTypeConverter &typeConverter, RewritePatternSet &patterns) |
| Populate patterns to eliminate bufferize materializations. More... | |
| LogicalResult | bufferizeOp (Operation *op, const AnalysisState &analysisState) |
Bufferize op and its nested ops that implement BufferizableOpInterface. More... | |
| LogicalResult | bufferizeOp (Operation *op, const BufferizationOptions &options) |
Bufferize op and its nested ops that implement BufferizableOpInterface. More... | |
| BufferizationOptions | getPartialBufferizationOptions () |
| LogicalResult | bufferizeOp (Operation *op, BufferizationState &bufferizationState) |
Bufferize op and its nested ops that implement BufferizableOpInterface. More... | |
| LogicalResult | finalizeBuffers (Operation *op, const BufferizationOptions &options) |
| Finalize all buffer allocations. More... | |
| FailureOr< memref::GlobalOp > | getGlobalFor (arith::ConstantOp constantOp, uint64_t alignment) |
| LogicalResult | analyzeOp (Operation *op, OneShotAnalysisState &state) |
Analyze op and its nested ops. More... | |
| LogicalResult | runOneShotBufferize (Operation *op, const OneShotBufferizationOptions &options) |
| Run One-Shot Bufferize on the given op: Analysis + Bufferization. More... | |
| LogicalResult | runOneShotModuleBufferize (ModuleOp moduleOp, bufferization::OneShotBufferizationOptions options) |
| Run One-Shot Module Bufferization on the given module. More... | |
| std::unique_ptr< Pass > | createBufferDeallocationPass () |
| Creates an instance of the BufferDeallocation pass to free all allocated buffers. More... | |
| LogicalResult | deallocateBuffers (Operation *op) |
| Run buffer deallocation. More... | |
| std::unique_ptr< Pass > | createBufferHoistingPass () |
| Creates a pass that moves allocations upwards to reduce the number of required copies that are inserted during the BufferDeallocation pass. More... | |
| std::unique_ptr< Pass > | createBufferLoopHoistingPass () |
| Creates a pass that moves allocations upwards out of loops. More... | |
| std::unique_ptr< Pass > | createBufferResultsToOutParamsPass () |
| Creates a pass that converts memref function results to out-params. More... | |
| LogicalResult | promoteBufferResultsToOutParams (ModuleOp module) |
| Replace buffers that are returned from a function with an out parameter. More... | |
| std::unique_ptr< OperationPass< func::FuncOp > > | createFinalizingBufferizePass () |
| Creates a pass that finalizes a partial bufferization by removing remaining bufferization.to_tensor and bufferization.to_memref operations. More... | |
| std::unique_ptr< Pass > | createOneShotBufferizePass () |
| Create a pass that bufferizes all ops that implement BufferizableOpInterface with One-Shot Bufferize. More... | |
| std::unique_ptr< Pass > | createOneShotBufferizePass (const OneShotBufferizationOptions &options) |
| Create a pass that bufferizes all ops that implement BufferizableOpInterface with One-Shot Bufferize and the specified bufferization options. More... | |
| std::unique_ptr< Pass > | createPromoteBuffersToStackPass (unsigned maxAllocSizeInBytes=1024, unsigned maxRankOfAllocatedMemRef=1) |
| Creates a pass that promotes heap-based allocations to stack-based ones. More... | |
| std::unique_ptr< Pass > | createPromoteBuffersToStackPass (std::function< bool(Value)> isSmallAlloc) |
| Creates a pass that promotes heap-based allocations to stack-based ones. More... | |
| std::unique_ptr< Pass > | createAllocTensorEliminationPass () |
| Create a pass that tries to eliminate alloc_tensor ops that are anchored on insert_slice ops. More... | |
| std::unique_ptr< Pass > | createBufferizationBufferizePass () |
| Create a pass that bufferizes ops from the bufferization dialect. More... | |
| void | registerAllocationOpInterfaceExternalModels (DialectRegistry ®istry) |
| Register external models for AllocationOpInterface. More... | |
| using mlir::bufferization::AnchorMatchFn = typedef std::function<bool(OpOperand &, SmallVector<Value> &)> |
A function that matches anchor OpOperands for AllocTensorOp elimination.
If an OpOperand is matched, the function should populate the SmallVector with all values that are needed during RewriteFn to produce the replacement value.
Definition at line 21 of file AllocTensorElimination.h.
| using mlir::bufferization::PostAnalysisStepFn = typedef std::function<LogicalResult( Operation *, AnalysisState &, BufferizationAliasInfo &, SmallVector<Operation *> &)> |
PostAnalysisStepFns can be registered with BufferizationOptions and are executed after the analysis, but before bufferization.
They can be used to implement custom dialect-specific optimizations. They may modify the IR, but must keep aliasInfo consistent. Newly created operations and operations that should be re-analyzed must be added to newOps.
Definition at line 30 of file OneShotAnalysis.h.
| using mlir::bufferization::PostAnalysisStepList = typedef SmallVector<PostAnalysisStepFn> |
Definition at line 32 of file OneShotAnalysis.h.
| using mlir::bufferization::RewriteFn = typedef std::function<Value(OpBuilder &, Location, OpOperand &)> |
A function that rewrites matched anchors.
Definition at line 24 of file AllocTensorElimination.h.
|
strong |
Specify fine-grain relationship between buffers to enable more analysis.
| Enumerator | |
|---|---|
| None | |
| Equivalent | |
Definition at line 333 of file BufferizableOpInterface.h.
| LogicalResult mlir::bufferization::analyzeOp | ( | Operation * | op, |
| OneShotAnalysisState & | state | ||
| ) |
Analyze op and its nested ops.
Bufferization decisions are stored in state.
Definition at line 917 of file OneShotAnalysis.cpp.
References annotateOpsWithBufferizationMarkers(), assertDestinationPassingStyle(), checkAliasInfoConsistency(), equivalenceAnalysis(), mlir::failed(), mlir::failure(), mlir::bufferization::OneShotAnalysisState::gatherUndefinedTensorUses(), mlir::bufferization::OneShotAnalysisState::gatherYieldedTensors(), mlir::bufferization::OneShotAnalysisState::getAliasInfo(), mlir::bufferization::AnalysisState::getOptions(), mlir::bufferization::AnalysisState::hasDialectState(), inPlaceAnalysis(), mlir::success(), and mlir::Operation::walk().
Referenced by mlir::bufferization::OneShotAnalysisState::getAliasInfo(), insertSliceAnchoredAllocTensorEliminationStep(), runOneShotBufferize(), and runOneShotModuleBufferize().
| LogicalResult mlir::bufferization::bufferizeOp | ( | Operation * | op, |
| const AnalysisState & | analysisState | ||
| ) |
Bufferize op and its nested ops that implement BufferizableOpInterface.
Whether buffer copies are needed or not is queried from state.
Note: If allowUnknownOps is set to false, bufferization fails when an unknown op (that does not implement BufferizableOpInterface) is found. No to_tensor/to_memref ops are inserted in that case.
Note: The layout map chosen to bufferize is the most dynamic canonical strided layout of the proper rank. This ensures compatibility with expected layouts after transformations. Combinations of memref.cast + canonicalization are responsible for clean ups.
Definition at line 330 of file Bufferize.cpp.
References mlir::bufferization::BufferizationOptions::bufferizeFunctionBoundaries, mlir::failed(), mlir::failure(), finalizeBuffers(), mlir::bufferization::AnalysisState::getOptions(), mlir::bufferization::AnalysisState::hasDialectState(), hasTensorSemantics(), mlir::bufferization::BufferizationOptions::isOpAllowed(), mlir::OpBuilder::Listener::notifyOperationInserted(), mlir::RewriterBase::notifyOperationRemoved(), options, and mlir::success().
Referenced by bufferizeOp(), populateEliminateBufferizeMaterializationsPatterns(), runOneShotBufferize(), and runOneShotModuleBufferize().
| LogicalResult mlir::bufferization::bufferizeOp | ( | Operation * | op, |
| const BufferizationOptions & | options | ||
| ) |
Bufferize op and its nested ops that implement BufferizableOpInterface.
Buffers are duplicated and copied before any tensor use that bufferizes to a memory write.
Note: This function bufferizes ops without utilizing analysis results. It can be used to implement partial bufferization passes.
Definition at line 507 of file Bufferize.cpp.
References bufferizeOp().
| LogicalResult mlir::bufferization::bufferizeOp | ( | Operation * | op, |
| BufferizationState & | bufferizationState | ||
| ) |
Bufferize op and its nested ops that implement BufferizableOpInterface.
Reuse an existing BufferizationState.
Note: This function overload is useful for extending the bufferization.
Check the result of bufferization. Return an error if an op was not bufferized, unless partial bufferization is allowed.
Definition at line 398 of file Bufferize.cpp.
References mlir::bufferization::BufferizationOptions::allowUnknownOps, mlir::Operation::emitError(), mlir::failed(), foldToMemrefToTensorPair(), mlir::Operation::getContext(), mlir::bufferization::BufferizationState::getOptions(), mlir::Operation::getUses(), hasTensorSemantics(), mlir::bufferization::BufferizationOptions::InferLayoutMap, isInPlace(), options, mlir::PreOrder, mlir::success(), and mlir::Operation::walk().
| FailureOr< Value > mlir::bufferization::castOrReallocMemRefValue | ( | OpBuilder & | b, |
| Value | value, | ||
| MemRefType | type | ||
| ) |
Try to cast the given ranked MemRef-typed value to the given ranked MemRef type.
Insert a reallocation + copy if it cannot be statically guaranteed that a direct cast would be valid.
E.g., when casting from a ranked MemRef type with dynamic layout to a ranked MemRef type with static layout, it is not statically known whether the cast will succeed or not. Such memref.cast ops may fail at runtime. This function never generates such casts and conservatively inserts a copy.
This function returns failure() in case of unsupported casts. E.g., casts with differing element types or memory spaces.
Definition at line 25 of file BufferizationOps.cpp.
References mlir::Type::cast(), copy(), mlir::OpBuilder::create(), mlir::OpBuilder::createOrFold(), mlir::failed(), mlir::failure(), mlir::Value::getLoc(), mlir::getStridesAndOffset(), mlir::Value::getType(), and value.
Referenced by mlir::bufferization::BufferizeTypeConverter::BufferizeTypeConverter(), and foldToMemrefToTensorPair().
| LogicalResult mlir::bufferization::createAllocDeallocOps | ( | Operation * | op, |
| const BufferizationOptions & | options, | ||
| bool | onlyLeakingAllocs = false, |
||
| bool * | changed = nullptr |
||
| ) |
Create alloc/dealloc ops as specified in the bufferization options.
If onlyLeakingAlloc, only those buffer allocations are processed for which no buffer deallocation can be created. changed is set to true if the IR was modified.
Definition at line 539 of file BufferizableOpInterface.cpp.
References mlir::WalkResult::advance(), mlir::bufferization::BufferizationOptions::createAlloc(), mlir::bufferization::BufferizationOptions::createDealloc(), mlir::failed(), mlir::Operation::getContext(), mlir::Block::getTerminator(), mlir::WalkResult::interrupt(), kBufferAllocationAttr, kSkipDeallocAttr, mlir::WalkResult::skip(), mlir::success(), mlir::Operation::walk(), and mlir::WalkResult::wasInterrupted().
Referenced by finalizeBuffers(), and replaceOpWithNewBufferizedOp().
| std::unique_ptr< Pass > mlir::bufferization::createAllocTensorEliminationPass | ( | ) |
Create a pass that tries to eliminate alloc_tensor ops that are anchored on insert_slice ops.
Definition at line 270 of file AllocTensorElimination.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createBufferDeallocationPass | ( | ) |
Creates an instance of the BufferDeallocation pass to free all allocated buffers.
Definition at line 702 of file BufferDeallocation.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createBufferHoistingPass | ( | ) |
Creates a pass that moves allocations upwards to reduce the number of required copies that are inserted during the BufferDeallocation pass.
Definition at line 420 of file BufferOptimizations.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createBufferizationBufferizePass | ( | ) |
Create a pass that bufferizes ops from the bufferization dialect.
Definition at line 258 of file Bufferize.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createBufferLoopHoistingPass | ( | ) |
Creates a pass that moves allocations upwards out of loops.
This avoids reallocations inside of loops.
Definition at line 424 of file BufferOptimizations.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createBufferResultsToOutParamsPass | ( | ) |
Creates a pass that converts memref function results to out-params.
Definition at line 193 of file BufferResultsToOutParams.cpp.
| std::unique_ptr< OperationPass< func::FuncOp > > mlir::bufferization::createFinalizingBufferizePass | ( | ) |
Creates a pass that finalizes a partial bufferization by removing remaining bufferization.to_tensor and bufferization.to_memref operations.
Definition at line 272 of file Bufferize.cpp.
Referenced by mlir::sparse_tensor::buildSparseCompiler().
| std::unique_ptr< Pass > mlir::bufferization::createOneShotBufferizePass | ( | ) |
Create a pass that bufferizes all ops that implement BufferizableOpInterface with One-Shot Bufferize.
Definition at line 262 of file Bufferize.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createOneShotBufferizePass | ( | const OneShotBufferizationOptions & | options | ) |
Create a pass that bufferizes all ops that implement BufferizableOpInterface with One-Shot Bufferize and the specified bufferization options.
Definition at line 266 of file Bufferize.cpp.
References options.
| std::unique_ptr< Pass > mlir::bufferization::createPromoteBuffersToStackPass | ( | unsigned | maxAllocSizeInBytes = 1024, |
| unsigned | maxRankOfAllocatedMemRef = 1 |
||
| ) |
Creates a pass that promotes heap-based allocations to stack-based ones.
Only buffers smaller than the provided size are promoted. Dynamic shaped buffers are promoted up to the given rank.
Definition at line 428 of file BufferOptimizations.cpp.
| std::unique_ptr< Pass > mlir::bufferization::createPromoteBuffersToStackPass | ( | std::function< bool(Value)> | isSmallAlloc | ) |
Creates a pass that promotes heap-based allocations to stack-based ones.
Only buffers smaller with isSmallAlloc(alloc) == true are promoted.
Definition at line 434 of file BufferOptimizations.cpp.
| LogicalResult mlir::bufferization::deallocateBuffers | ( | Operation * | op | ) |
Run buffer deallocation.
Definition at line 655 of file BufferDeallocation.cpp.
References mlir::WalkResult::advance(), mlir::Operation::emitError(), mlir::failed(), mlir::failure(), mlir::WalkResult::interrupt(), mlir::success(), validateSupportedControlFlow(), mlir::Operation::walk(), and mlir::WalkResult::wasInterrupted().
Referenced by finalizeBuffers(), and validateSupportedControlFlow().
| LogicalResult mlir::bufferization::eliminateAllocTensors | ( | RewriterBase & | rewriter, |
| Operation * | op, | ||
| bufferization::AnalysisState & | state, | ||
| AnchorMatchFn | anchorMatchFunc, | ||
| RewriteFn | rewriteFunc | ||
| ) |
Try to eliminate AllocTensorOps inside op.
rewriteFunc generates the replacement for the AllocTensorOp.Only AllocTensorOps that are anchored on a matching OpOperand as per anchorMatchFunc are considered. "Anchored" means that there is a path on the reverse SSA use-def chain, starting from the OpOperand and always following the aliasing OpOperand, that eventually ends at a single AllocTensorOp.
An AllocTensorOp is replaced with the result of rewriteFunc if it is anchored on a matching OpOperand. "Anchored" means that there is a path on the reverse SSA use-def chain, starting from the OpOperand and always following the aliasing OpOperand, that eventually ends at a single AllocTensorOp.
Definition at line 101 of file AllocTensorElimination.cpp.
References mlir::WalkResult::advance(), mlir::bufferization::AnalysisState::areEquivalentBufferizedValues(), mlir::Value::dyn_cast(), mlir::failure(), findValidInsertionPoint(), mlir::bufferization::AnalysisState::findValueInReverseUseDefChain(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::bufferization::AnalysisState::getAliasingOpOperand(), mlir::Value::getDefiningOp(), mlir::Value::getLoc(), mlir::Operation::getOpOperands(), mlir::bufferization::AnalysisState::isInPlace(), mlir::RewriterBase::replaceOp(), mlir::OpBuilder::setInsertionPoint(), mlir::WalkResult::skip(), mlir::Operation::walk(), and mlir::WalkResult::wasInterrupted().
Referenced by insertSliceAnchoredAllocTensorEliminationStep().
| LogicalResult mlir::bufferization::finalizeBuffers | ( | Operation * | op, |
| const BufferizationOptions & | options | ||
| ) |
Finalize all buffer allocations.
Definition at line 296 of file Bufferize.cpp.
References createAllocDeallocOps(), mlir::bufferization::BufferizationOptions::createDeallocs, deallocateBuffers(), mlir::failed(), mlir::failure(), hoistBufferAllocations(), promoteBufferResultsToOutParams(), mlir::bufferization::BufferizationOptions::promoteBufferResultsToOutParams, and mlir::success().
Referenced by bufferizeOp(), and runOneShotModuleBufferize().
| LogicalResult mlir::bufferization::foldToMemrefToTensorPair | ( | RewriterBase & | rewriter, |
| ToMemrefOp | toMemref, | ||
| bool | allowSameType = true |
||
| ) |
Try to fold to_memref(to_tensor(x)).
If x's type and the result type of the to_memref op are different, a memref.cast is needed.
Definition at line 85 of file BufferizationOps.cpp.
References mlir::RewritePatternSet::add(), castOrReallocMemRefValue(), mlir::OpBuilder::create(), mlir::bufferization::BufferizationState::createAlloc(), mlir::Type::dyn_cast(), mlir::emitError(), mlir::RewriterBase::eraseOp(), mlir::failed(), mlir::failure(), mlir::memref::findDealloc(), mlir::memref::foldMemRefCast(), mlir::SideEffects::Effect::Base< DerivedEffect, BaseEffect >::get(), mlir::SideEffects::Resource::Base< DefaultResource >::get(), mlir::Operation::getBlock(), mlir::Value::getLoc(), mlir::m_ConstantInt(), mlir::matchPattern(), mlir::RewriterBase::replaceOp(), replaceOpWithBufferizedValues(), mlir::RewriterBase::replaceOpWithNewOp(), mlir::succeeded(), mlir::success(), value, and mlir::verify().
Referenced by bufferizeOp().
| FailureOr< memref::GlobalOp > mlir::bufferization::getGlobalFor | ( | arith::ConstantOp | constantOp, |
| uint64_t | alignment | ||
| ) |
Definition at line 149 of file BufferUtils.cpp.
References mlir::TypeConverter::convertType(), mlir::failure(), mlir::Region::getOps(), and mlir::Operation::getRegion().
| BaseMemRefType mlir::bufferization::getMemRefType | ( | TensorType | tensorType, |
| const BufferizationOptions & | options, | ||
| MemRefLayoutAttrInterface | layout = {}, |
||
| Attribute | memorySpace = {} |
||
| ) |
Return a MemRefType to which the tensorType can be bufferized.
If possible, op bufferization implementations should not use this function and instead infer precise memref types for tensor results by themselves.
Unless a layout map was specified, options.unknownTypeConverter determines what kind of layout map will be used. For best composability (without copies), the fully dynamic layout map is used by default.
Note: Canonicalization patterns could clean up layout maps and infer more precise layout maps after bufferization. However, many possible canonicalizations are currently not implemented.
Definition at line 653 of file BufferizableOpInterface.cpp.
References mlir::Type::cast(), mlir::Type::dyn_cast(), mlir::bufferization::BufferizationOptions::FullyDynamicLayoutMap, getMemRefTypeWithFullyDynamicLayout(), getMemRefTypeWithStaticIdentityLayout(), mlir::bufferization::BufferizationOptions::IdentityLayoutMap, and mlir::bufferization::BufferizationOptions::unknownTypeConversion.
Referenced by mlir::bufferization::func_ext::FuncOpInterface::bufferize(), foldTransferInBoundsAttribute(), mlir::bufferization::BufferizationState::getBufferType(), lookupBuffer(), CollapseShapeOpMemRefCastFolder::matchAndRewrite(), CanonicalizeSingleResultAffineMinMaxOp< T >::matchAndRewrite(), parseGlobalMemrefOpTypeAndInitialValue(), mlir::replaceForOpWithNewYields(), replaceOpWithNewBufferizedOp(), verifyMemoryOpIndexing(), verifyMultShape(), and verifyVectorMemoryOp().
| BaseMemRefType mlir::bufferization::getMemRefTypeWithFullyDynamicLayout | ( | TensorType | tensorType, |
| Attribute | memorySpace = {} |
||
| ) |
Return a MemRef type with fully dynamic layout.
If the given tensor type is unranked, return an unranked MemRef type.
Definition at line 686 of file BufferizableOpInterface.cpp.
References mlir::Type::cast(), mlir::Type::dyn_cast(), and mlir::makeStridedLinearLayoutMap().
Referenced by mlir::bufferization::func_ext::FuncOpInterface::bufferize(), mlir::bufferization::func_ext::getBufferizedFunctionArgType(), getMemRefType(), and replaceOpWithNewBufferizedOp().
| BaseMemRefType mlir::bufferization::getMemRefTypeWithStaticIdentityLayout | ( | TensorType | tensorType, |
| Attribute | memorySpace = {} |
||
| ) |
Return a MemRef type with a static identity layout (i.e., no layout map).
If the given tensor type is unranked, return an unranked MemRef type.
Definition at line 709 of file BufferizableOpInterface.cpp.
References mlir::Type::cast(), and mlir::Type::dyn_cast().
Referenced by mlir::bufferization::func_ext::FuncOpInterface::bufferize(), mlir::bufferization::func_ext::getBufferizedFunctionArgType(), getMemRefType(), and replaceOpWithNewBufferizedOp().
| BufferizationOptions mlir::bufferization::getPartialBufferizationOptions | ( | ) |
Definition at line 513 of file Bufferize.cpp.
References mlir::bufferization::BufferizationOptions::allowUnknownOps, mlir::bufferization::BufferizationOptions::createDeallocs, mlir::bufferization::BufferizationOptions::IdentityLayoutMap, options, and mlir::bufferization::BufferizationOptions::unknownTypeConversion.
Referenced by populateEliminateBufferizeMaterializationsPatterns().
| LogicalResult mlir::bufferization::hoistBufferAllocations | ( | Operation * | op, |
| const BufferizationOptions & | options | ||
| ) |
Try to hoist all new buffer allocations until the next hoisting barrier.
Definition at line 588 of file BufferizableOpInterface.cpp.
References mlir::Block::findAncestorOpInBlock(), mlir::Operation::getParentOp(), mlir::Operation::getRegions(), mlir::bufferization::BufferizationOptions::hoistAllocations, kBufferAllocationAttr, mlir::success(), and mlir::Operation::walk().
Referenced by finalizeBuffers(), and replaceOpWithNewBufferizedOp().
| LogicalResult mlir::bufferization::insertSliceAnchoredAllocTensorEliminationStep | ( | RewriterBase & | rewriter, |
| Operation * | op, | ||
| bufferization::AnalysisState & | state | ||
| ) |
Try to eliminate AllocTensorOps inside op that are anchored on an InsertSliceOp, i.e., if it is eventually inserted into another tensor (and some other conditions are met).
Try to eliminate AllocTensorOps inside op.
An AllocTensorOp can be eliminated if it is eventually inserted into another tensor (and some other conditions are met).
E.g.: %0 = linalg.alloc_tensor %1 = linalg.fill(cst, %0) {inplace = [true]} %2 = tensor.insert_slice %1 into t[10][20][1]
AllocTensorOp elimination will try to fill t inplace instead of filling a new allocation %0 and inserting it into t. This is done by replacing the AllocTensorOp with:
%0 = tensor.extract_slice t[10][20][1]
The analysis looks for matching ExtractSliceOp/InsertSliceOp pairs and lets those bufferize inplace in the absence of other conflicts.
Starting from an InsertSliceOp, an AllocTensorOp at the end of the insert source's reverse use-def chain is eliminated if:
Definition at line 191 of file AllocTensorElimination.cpp.
References analyzeOp(), mlir::OpBuilder::create(), eliminateAllocTensors(), mlir::failed(), mlir::Operation::getContext(), mlir::Builder::getIndexAttr(), mlir::detail::IROperandBase::getOwner(), mlir::DialectRegistry::insert(), and options.
| bool mlir::bufferization::isFunctionArgument | ( | Value | value | ) |
Return true if the given value is a BlockArgument of a func::FuncOp.
Definition at line 646 of file BufferizableOpInterface.cpp.
References mlir::Value::dyn_cast().
| Value mlir::bufferization::lookupBuffer | ( | RewriterBase & | rewriter, |
| Value | tensor, | ||
| const BufferizationOptions & | options | ||
| ) |
Lookup the buffer for the given value.
If the value was not bufferized yet, wrap it in a ToMemrefOp. Otherwise, it is the result of a ToTensorOp, from which the memref operand is returned.
Note: Use BufferizationState::getBuffer during bufferization. lookupBuffer is just for compatibility and gradual migration of bufferization patterns to BufferizableOpInterface-based bufferization. It does not insert any buffer copies.
Definition at line 255 of file BufferizableOpInterface.cpp.
References mlir::OpBuilder::create(), mlir::Type::dyn_cast(), ensureToMemrefOpIsValid(), mlir::Value::getDefiningOp(), mlir::Value::getLoc(), getMemRefType(), mlir::Value::getType(), and setInsertionPointAfter().
Referenced by mlir::bufferization::BufferizationState::getAnalysisState(), and mlir::bufferization::BufferizationState::getBuffer().
| void mlir::bufferization::populateBufferizeMaterializationLegality | ( | ConversionTarget & | target | ) |
Marks ops used by bufferization for type conversion materializations as "legal" in the given ConversionTarget.
This function should be called by all bufferization passes using BufferizeTypeConverter so that materializations work properly. One exception is bufferization passes doing "full" conversions, where it can be desirable for even the materializations to remain illegal so that they are eliminated, such as via the patterns in populateEliminateBufferizeMaterializationsPatterns.
Definition at line 79 of file Bufferize.cpp.
References mlir::ConversionTarget::addLegalOp(), mlir::OpConversionPattern< SourceOp >::OpConversionPattern(), mlir::ConversionPatternRewriter::replaceOp(), and mlir::success().
| void mlir::bufferization::populateEliminateBufferizeMaterializationsPatterns | ( | BufferizeTypeConverter & | typeConverter, |
| RewritePatternSet & | patterns | ||
| ) |
Populate patterns to eliminate bufferize materializations.
In particular, these are the tensor_load/buffer_cast ops.
Definition at line 116 of file Bufferize.cpp.
References mlir::RewritePatternSet::add(), mlir::OpPassManager::addPass(), mlir::bufferization::BufferizationOptions::allowDialectInFilter(), mlir::bufferization::BufferizationOptions::allowOperationInFilter(), mlir::bufferization::OneShotBufferizationOptions::allowReturnAllocs, mlir::bufferization::BufferizationOptions::allowUnknownOps, mlir::bufferization::BufferizationOptions::alwaysAliasingWithDest, mlir::bufferization::BufferizationOptions::analysisFuzzerSeed, mlir::applyFullConversion(), mlir::bufferization::BufferizationOptions::bufferizeFunctionBoundaries, bufferizeOp(), mlir::createCanonicalizerPass(), mlir::createCSEPass(), mlir::bufferization::BufferizationOptions::createDeallocs, mlir::createLoopInvariantCodeMotionPass(), mlir::bufferization::OneShotBufferizationOptions::dropEquivalentFuncResults, mlir::failed(), mlir::bufferization::BufferizationOptions::FullyDynamicLayoutMap, mlir::bufferization::BufferizationOptions::functionBoundaryTypeConversion, mlir::RewritePatternSet::getContext(), getPartialBufferizationOptions(), mlir::bufferization::BufferizationOptions::IdentityLayoutMap, mlir::bufferization::BufferizationOptions::InferLayoutMap, mlir::DialectRegistry::insert(), mlir::TypeConverter::isLegal(), mlir::ConversionTarget::markUnknownOpDynamicallyLegal(), options, mlir::bufferization::BufferizationOptions::printConflicts, promoteBufferResultsToOutParams(), mlir::bufferization::BufferizationOptions::promoteBufferResultsToOutParams, registerAllocationOpInterfaceExternalModels(), runOneShotBufferize(), runOneShotModuleBufferize(), mlir::bufferization::BufferizationOptions::testAnalysisOnly, and mlir::bufferization::BufferizationOptions::unknownTypeConversion.
| LogicalResult mlir::bufferization::promoteBufferResultsToOutParams | ( | ModuleOp | module | ) |
Replace buffers that are returned from a function with an out parameter.
Also update all call sites.
Definition at line 168 of file BufferResultsToOutParams.cpp.
References mlir::failed(), mlir::failure(), mlir::success(), updateCalls(), updateFuncOp(), and updateReturnOps().
Referenced by finalizeBuffers(), and populateEliminateBufferizeMaterializationsPatterns().
| void mlir::bufferization::registerAllocationOpInterfaceExternalModels | ( | DialectRegistry & | registry | ) |
Register external models for AllocationOpInterface.
Definition at line 691 of file BufferDeallocation.cpp.
References mlir::DialectRegistry::addExtension().
Referenced by populateEliminateBufferizeMaterializationsPatterns(), and validateSupportedControlFlow().
| void mlir::bufferization::replaceOpWithBufferizedValues | ( | RewriterBase & | rewriter, |
| Operation * | op, | ||
| ValueRange | values | ||
| ) |
Replace an op with replacement values.
The op is deleted. Tensor OpResults must be replaced with memref values.
Definition at line 347 of file BufferizableOpInterface.cpp.
References mlir::OpBuilder::create(), mlir::Value::getLoc(), mlir::Operation::getNumResults(), mlir::Operation::getOpResults(), mlir::Value::getType(), mlir::Type::isa(), mlir::RewriterBase::replaceOp(), and mlir::OpBuilder::setInsertionPointAfter().
Referenced by mlir::bufferization::func_ext::CallOpInterface::bufferize(), foldToMemrefToTensorPair(), mlir::bufferization::BufferizationState::getAnalysisState(), and replaceOpWithNewBufferizedOp().
| OpTy mlir::bufferization::replaceOpWithNewBufferizedOp | ( | RewriterBase & | rewriter, |
| Operation * | op, | ||
| Args &&... | args | ||
| ) |
Replace an op with a new op.
The new op must have the same number of results as the replaced op. The new op may not return any tensor values.
Definition at line 586 of file BufferizableOpInterface.h.
References mlir::OpBuilder::create(), createAllocDeallocOps(), mlir::Operation::getLoc(), getMemRefType(), getMemRefTypeWithFullyDynamicLayout(), getMemRefTypeWithStaticIdentityLayout(), mlir::Operation::getResults(), hoistBufferAllocations(), and replaceOpWithBufferizedValues().
| LogicalResult mlir::bufferization::runOneShotBufferize | ( | Operation * | op, |
| const OneShotBufferizationOptions & | options | ||
| ) |
Run One-Shot Bufferize on the given op: Analysis + Bufferization.
Definition at line 976 of file OneShotAnalysis.cpp.
References analyzeOp(), bufferizeOp(), mlir::failed(), mlir::failure(), mlir::success(), and mlir::bufferization::BufferizationOptions::testAnalysisOnly.
Referenced by mlir::bufferization::OneShotAnalysisState::getAliasInfo(), and populateEliminateBufferizeMaterializationsPatterns().
| LogicalResult mlir::bufferization::runOneShotModuleBufferize | ( | ModuleOp | moduleOp, |
| bufferization::OneShotBufferizationOptions | options | ||
| ) |
Run One-Shot Module Bufferization on the given module.
Performs a simple function call analysis to determine which function arguments are inplaceable. Then analyzes and bufferizes FuncOps one-by-one with One-Shot Bufferize.
Definition at line 418 of file OneShotModuleBufferize.cpp.
References mlir::bufferization::OneShotBufferizationOptions::addPostAnalysisStep(), mlir::bufferization::OneShotBufferizationOptions::allowReturnAllocs, mlir::bufferization::func_ext::Analyzed, mlir::bufferization::func_ext::FuncAnalysisState::analyzedFuncOps, analyzeOp(), annotateOpsWithBufferizationMarkers(), mlir::bufferization::BufferizationOptions::bufferizeFunctionBoundaries, bufferizeOp(), equivalenceAnalysis(), mlir::failed(), mlir::failure(), finalizeBuffers(), foldMemRefCasts(), mlir::bufferization::BufferizationOptions::functionBoundaryTypeConversion, mlir::bufferization::func_ext::getFuncAnalysisState(), getFuncOpsOrderedByCalls(), mlir::bufferization::BufferizationOptions::InferLayoutMap, removeBufferizationAttributes(), mlir::bufferization::func_ext::FuncAnalysisState::startFunctionAnalysis(), mlir::success(), and mlir::bufferization::BufferizationOptions::testAnalysisOnly.
Referenced by populateEliminateBufferizeMaterializationsPatterns().
1.8.13