Classes
struct	ContinuousTileSizeSpecificationBase
struct	MultiSizeSpecificationBase
struct	PackingResult
	Helper struct to hold the results of building a packing loop nest. More...

Enumerations
enum class	MatchContractionResult { Success = 0 , NotLinalgOp , WrongNumOperands , NoReduction , NotProjectedPermutations , NotAddMul }
enum class	MatchConvolutionResult { Success = 0 , NotLinalgOp , WrongNumOperands , WrongInputIndexingMap , NotProjectedPermutations , NonConvolutionLoop , OutputDimsNotParallel , NonOutputDimNotReduction , EmptyConvolvedDims }

Functions
bool	canOpOperandsBeDroppedImpl (linalg::LinalgOp linalgOp, ArrayRef< OpOperand * > droppedOperands)
	Implementation of the method that check if given operands can be dropped, i.e.
bool	isContractionBody (Block &block, function_ref< bool(Operation , Operation )> isaPair, llvm::raw_ostream &errs=mlir::thread_safe_nulls())
	Returns true if the block contains a contraction of the following form:
MatchContractionResult	isContractionInterfaceImpl (Operation op, ContractionDimensions dimensions=nullptr)
	Checks whether op conforms to ContractionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.
StringRef	getMatchContractionMessage (MatchContractionResult res)
	Returns the error message corresponding to the contraction checking return code.
MatchConvolutionResult	isConvolutionInterfaceImpl (Operation op, ConvolutionDimensions dimensions=nullptr, bool allowEmptyConvolvedDims=false)
	Checks whether op conforms to ConvolutionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.
StringRef	getMatchConvolutionMessage (MatchConvolutionResult res)
	Returns the error message corresponding to the convolution checking return code.
LogicalResult	verifyContractionInterface (Operation *op)
	Verify that op conforms to ContractionOpInterface.
LogicalResult	verifyConvolutionInterface (Operation *op)
	Verify that op conforms to the ConvolutionOpInterface.
LogicalResult	verifyFillInterface (Operation *op)
	Verify that op conforms to the FillOpInterface.
LogicalResult	verifyStructuredOpInterface (Operation *op)
	Verify that op conforms to the invariants of StructuredOpInterface.
FailureOr< PackingResult >	buildPackingLoopNest (RewriterBase &rewriter, tensor::PadOp opToHoist, scf::ForOp outermostEnclosingForOp, ArrayRef< int64_t > transposeVector)
	Build the packing loop nest required to hoist opToHoist above outermostEnclosingForOp.

Enumeration Type Documentation

◆ MatchContractionResult

enum class mlir::linalg::detail::MatchContractionResult

strong

Enumerator
Success
NotLinalgOp
WrongNumOperands
NoReduction
NotProjectedPermutations
NotAddMul

Definition at line 502 of file LinalgInterfaces.cpp.

◆ MatchConvolutionResult

enum class mlir::linalg::detail::MatchConvolutionResult

strong

Enumerator
Success
NotLinalgOp
WrongNumOperands
WrongInputIndexingMap
NotProjectedPermutations
NonConvolutionLoop
OutputDimsNotParallel
NonOutputDimNotReduction
EmptyConvolvedDims

Definition at line 870 of file LinalgInterfaces.cpp.

Function Documentation

◆ buildPackingLoopNest()

FailureOr< PackingResult > mlir::linalg::detail::buildPackingLoopNest	(	RewriterBase &	rewriter,
		tensor::PadOp	opToHoist,
		scf::ForOp	outermostEnclosingForOp,
		ArrayRef< int64_t >	transposeVector )

Build the packing loop nest required to hoist opToHoist above outermostEnclosingForOp.

The loop nest is built just before outermostEnclosingForOp.

Definition at line 728 of file HoistPadding.cpp.

References buildPackingLoopNestImpl(), and DBGS.

◆ canOpOperandsBeDroppedImpl()

bool mlir::linalg::detail::canOpOperandsBeDroppedImpl	(	linalg::LinalgOp	linalgOp,
		ArrayRef< OpOperand * >	droppedOperands )

Implementation of the method that check if given operands can be dropped, i.e.

Include the definitions of the copy operation interface.

the remaining operands can compute the loop bounds of the op.

Definition at line 40 of file LinalgInterfaces.cpp.

References mlir::concatAffineMaps(), and mlir::inversePermutation().

◆ getMatchContractionMessage()

StringRef mlir::linalg::detail::getMatchContractionMessage ( MatchContractionResult res )

Returns the error message corresponding to the contraction checking return code.

Definition at line 547 of file LinalgInterfaces.cpp.

References NoReduction, NotAddMul, NotLinalgOp, NotProjectedPermutations, Success, and WrongNumOperands.

Referenced by verifyContractionInterface().

◆ getMatchConvolutionMessage()

StringRef mlir::linalg::detail::getMatchConvolutionMessage ( MatchConvolutionResult res )

Returns the error message corresponding to the convolution checking return code.

Definition at line 1018 of file LinalgInterfaces.cpp.

References EmptyConvolvedDims, NonConvolutionLoop, NonOutputDimNotReduction, NotLinalgOp, NotProjectedPermutations, OutputDimsNotParallel, Success, WrongInputIndexingMap, and WrongNumOperands.

Referenced by verifyConvolutionInterface().

◆ isContractionBody()

bool mlir::linalg::detail::isContractionBody	(	Block &	block,
		function_ref< bool(Operation , Operation )>	isaPair,
		llvm::raw_ostream &	errs = mlir::thread_safe_nulls() )

Returns true if the block contains a contraction of the following form:

%0 = <elemwise>(permutation-of(cu(block-argument-0), cu(block-argument-1))) %1 = <reduce>(permutation-of(cu(%0), cu(block-argument-2))) return-like cu(%1)

where <elemwise> and <reduce> are binary operations constituting a contraction (in the canonical case, <elemwise> is a multiplication and <reduce> is an addition). The name and other properties of these operations are checked by isaPair. All operands of all operations may be supplied through a chain of side effect-free unary operations, such as casts, which is denoted as cu above.

When the body does not contain a contraction, a more precise description of the failed precondition is send to the errs stream, if provided.

Definition at line 297 of file LinalgInterfaces.cpp.

References mlir::Block::back(), mlir::Block::empty(), mlir::Block::getArgument(), mlir::Value::getDefiningOp(), mlir::Block::getNumArguments(), mlir::Operation::getNumOperands(), mlir::Operation::getNumResults(), mlir::Operation::getOperand(), getSourceSkipUnary(), mlir::Block::getTerminator(), and mlir::Operation::mightHaveTrait().

Referenced by isContractionBody(), and isContractionInterfaceImpl().

◆ isContractionInterfaceImpl()

mlir::linalg::detail::MatchContractionResult mlir::linalg::detail::isContractionInterfaceImpl	(	Operation *	op,
		ContractionDimensions *	dimensions = nullptr )

Checks whether op conforms to ContractionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.

Definition at line 513 of file LinalgInterfaces.cpp.

References mlir::linalg::inferContractionDims(), isContractionBody(), mlir::AffineMap::isProjectedPermutation(), NoReduction, NotAddMul, NotLinalgOp, NotProjectedPermutations, Success, and WrongNumOperands.

Referenced by mlir::linalg::isaContractionOpInterface(), and verifyContractionInterface().

◆ isConvolutionInterfaceImpl()

mlir::linalg::detail::MatchConvolutionResult mlir::linalg::detail::isConvolutionInterfaceImpl	(	Operation *	op,
		ConvolutionDimensions *	dimensions = nullptr,
		bool	allowEmptyConvolvedDims = false )

Checks whether op conforms to ConvolutionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.

If allowEmptyConvolvedDims is not set, we further checks whether the op contains convolved dims.

Definition at line 884 of file LinalgInterfaces.cpp.

References EmptyConvolvedDims, getPreservedDims(), inferConvolutionDimsImpl(), NonConvolutionLoop, NonOutputDimNotReduction, NotLinalgOp, NotProjectedPermutations, OutputDimsNotParallel, Success, WrongInputIndexingMap, and WrongNumOperands.

Referenced by mlir::linalg::isaConvolutionOpInterface(), and verifyConvolutionInterface().

◆ verifyContractionInterface()

LogicalResult mlir::linalg::detail::verifyContractionInterface ( Operation * op )

Verify that op conforms to ContractionOpInterface.

Verify that a LinalgOp op is a contraction.

A Linalg contraction is defined in general terms:

Has 2 input and 1 output shapes.
Has at least one reduction dimension.
Has only projected permutation indexing maps.
its body computes u5(u1(c) + u2(u3(a) * u4(b))) on some field (AddOpType, MulOpType), where u1, u2, u3, u4 and u5 represent scalar unary operations that may change the type (e.g. for mixed-precision). As a consequence, when vectorization of such an op occurs, the only special behavior is that the (unique) MulOpType is vectorized into a vector.contract. All other ops are handled in a generic fashion. In the future, we may wish to allow more input arguments and elementwise and constant operations that do not involve the reduction dimension(s).

Definition at line 587 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), getMatchContractionMessage(), isContractionInterfaceImpl(), Success, and success().

◆ verifyConvolutionInterface()

LogicalResult mlir::linalg::detail::verifyConvolutionInterface ( Operation * op )

Verify that op conforms to the ConvolutionOpInterface.

Definition at line 1049 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), getMatchConvolutionMessage(), isConvolutionInterfaceImpl(), Success, and success().

◆ verifyFillInterface()

LogicalResult mlir::linalg::detail::verifyFillInterface ( Operation * op )

Verify that op conforms to the FillOpInterface.

Definition at line 1081 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), isFillInterfaceImpl(), NotLinalgOp, NotScalarInput, success(), and WrongNumOperands.

◆ verifyStructuredOpInterface()

LogicalResult mlir::linalg::detail::verifyStructuredOpInterface ( Operation * op )

Verify that op conforms to the invariants of StructuredOpInterface.

Definition at line 1242 of file LinalgInterfaces.cpp.

References mlir::Operation::emitOpError(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::Block::getArgument(), mlir::getElementTypeOrSelf(), mlir::Block::getNumArguments(), mlir::AffineMap::getNumDims(), mlir::Operation::getNumOperands(), mlir::OpOperand::getOperandNumber(), mlir::Value::getType(), and success().

Classes

Enumerations

Functions