Classes
struct	PackingResult
	Helper struct to hold the results of building a packing loop nest. More...

struct	MultiSizeSpecificationBase

struct	ContinuousTileSizeSpecificationBase

Enumerations
enum class	MatchContractionResult { Success = 0 , NotLinalgOp , WrongNumOperands , NoReduction , NotProjectedPermutations , NotAddMul }

enum class	MatchConvolutionResult { Success = 0 , NotLinalgOp , WrongNumOperands , WrongInputIndexingMap , NotProjectedPermutations , NonConvolutionLoop , OutputDimsNotParallel , NonOutputDimNotReduction , EmptyConvolvedDims }

Functions
bool	canOpOperandsBeDroppedImpl (linalg::LinalgOp linalgOp, ArrayRef< OpOperand * > droppedOperands)
	Implementation of the method that check if given operands can be dropped, i.e. More...

bool	isContractionBody (Block &block, function_ref< bool(Operation , Operation )> isaPair, llvm::raw_ostream &errs=mlir::thread_safe_nulls())
	Returns true if the block contains a contraction of the following form: More...

MatchContractionResult	isContractionInterfaceImpl (Operation op, ContractionDimensions dimensions=nullptr)
	Checks whether `op` conforms to ContractionOpInterface and populates `dimensions` with indexes of the different kinds of dimensions when present. More...

StringRef	getMatchContractionMessage (MatchContractionResult res)
	Returns the error message corresponding to the contraction checking return code. More...

MatchConvolutionResult	isConvolutionInterfaceImpl (Operation op, ConvolutionDimensions dimensions=nullptr, bool allowEmptyConvolvedDims=false)
	Checks whether `op` conforms to ConvolutionOpInterface and populates `dimensions` with indexes of the different kinds of dimensions when present. More...

StringRef	getMatchConvolutionMessage (MatchConvolutionResult res)
	Returns the error message corresponding to the convolution checking return code. More...

LogicalResult	verifyContractionInterface (Operation *op)
	Verify that `op` conforms to ContractionOpInterface. More...

LogicalResult	verifyConvolutionInterface (Operation *op)
	Verify that `op` conforms to the ConvolutionOpInterface. More...

LogicalResult	verifyFillInterface (Operation *op)
	Verify that `op` conforms to the FillOpInterface. More...

LogicalResult	verifyStructuredOpInterface (Operation *op)
	Verify that `op` conforms to the invariants of StructuredOpInterface. More...

FailureOr< PackingResult >	buildPackingLoopNest (RewriterBase &rewriter, tensor::PadOp opToHoist, scf::ForOp outermostEnclosingForOp, ArrayRef< int64_t > transposeVector)
	Build the packing loop nest required to hoist `opToHoist` above `outermostEnclosingForOp`. More...

Enumeration Type Documentation

◆ MatchContractionResult

enum mlir::linalg::detail::MatchContractionResult

strong

Enumerator
Success
NotLinalgOp
WrongNumOperands
NoReduction
NotProjectedPermutations
NotAddMul

Definition at line 465 of file LinalgInterfaces.cpp.

◆ MatchConvolutionResult

enum mlir::linalg::detail::MatchConvolutionResult

strong

Enumerator
Success
NotLinalgOp
WrongNumOperands
WrongInputIndexingMap
NotProjectedPermutations
NonConvolutionLoop
OutputDimsNotParallel
NonOutputDimNotReduction
EmptyConvolvedDims

Definition at line 833 of file LinalgInterfaces.cpp.

Function Documentation

◆ buildPackingLoopNest()

FailureOr< PackingResult > mlir::linalg::detail::buildPackingLoopNest	(	RewriterBase &	rewriter,
		tensor::PadOp	opToHoist,
		scf::ForOp	outermostEnclosingForOp,
		ArrayRef< int64_t >	transposeVector
	)

Build the packing loop nest required to hoist opToHoist above outermostEnclosingForOp.

The loop nest is built just before outermostEnclosingForOp.

Definition at line 722 of file HoistPadding.cpp.

References buildPackingLoopNestImpl(), and DBGS.

◆ canOpOperandsBeDroppedImpl()

bool mlir::linalg::detail::canOpOperandsBeDroppedImpl	(	linalg::LinalgOp	linalgOp,
		ArrayRef< OpOperand * >	droppedOperands
	)

Implementation of the method that check if given operands can be dropped, i.e.

Include the definitions of the copy operation interface.

the remaining operands can compute the loop bounds of the op.

Definition at line 42 of file LinalgInterfaces.cpp.

References mlir::concatAffineMaps(), and mlir::inversePermutation().

◆ getMatchContractionMessage()

StringRef mlir::linalg::detail::getMatchContractionMessage ( MatchContractionResult res )

Returns the error message corresponding to the contraction checking return code.

Definition at line 510 of file LinalgInterfaces.cpp.

Referenced by verifyContractionInterface().

◆ getMatchConvolutionMessage()

StringRef mlir::linalg::detail::getMatchConvolutionMessage ( MatchConvolutionResult res )

Returns the error message corresponding to the convolution checking return code.

Definition at line 981 of file LinalgInterfaces.cpp.

Referenced by verifyConvolutionInterface().

◆ isContractionBody()

bool mlir::linalg::detail::isContractionBody	(	Block &	block,
		function_ref< bool(Operation , Operation )>	isaPair,
		llvm::raw_ostream &	errs = `mlir::thread_safe_nulls()`
	)

Returns true if the block contains a contraction of the following form:

%0 = <elemwise>(permutation-of(cu(block-argument-0), cu(block-argument-1))) %1 = <reduce>(permutation-of(cu(%0), cu(block-argument-2))) return-like cu(%1)

where <elemwise> and <reduce> are binary operations constituting a contraction (in the canonical case, <elemwise> is a multiplication and <reduce> is an addition). The name and other properties of these operations are checked by isaPair. All operands of all operations may be supplied through a chain of side effect-free unary operations, such as casts, which is denoted as cu above.

When the body does not contain a contraction, a more precise description of the failed precondition is send to the errs stream, if provided.

Definition at line 261 of file LinalgInterfaces.cpp.

References mlir::Block::back(), mlir::Block::empty(), mlir::Block::getArgument(), mlir::Value::getDefiningOp(), mlir::Block::getNumArguments(), mlir::Operation::getNumOperands(), mlir::Operation::getNumResults(), mlir::Operation::getOperand(), getSourceSkipUnary(), mlir::Block::getTerminator(), and mlir::Operation::mightHaveTrait().

Referenced by isContractionBody().

◆ isContractionInterfaceImpl()

mlir::linalg::detail::MatchContractionResult mlir::linalg::detail::isContractionInterfaceImpl	(	Operation *	op,
		ContractionDimensions *	dimensions = `nullptr`
	)

Checks whether op conforms to ContractionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.

Definition at line 476 of file LinalgInterfaces.cpp.

Referenced by mlir::linalg::isaContractionOpInterface(), and verifyContractionInterface().

◆ isConvolutionInterfaceImpl()

mlir::linalg::detail::MatchConvolutionResult mlir::linalg::detail::isConvolutionInterfaceImpl	(	Operation *	op,
		ConvolutionDimensions *	dimensions = `nullptr`,
		bool	allowEmptyConvolvedDims = `false`
	)

Checks whether op conforms to ConvolutionOpInterface and populates dimensions with indexes of the different kinds of dimensions when present.

If allowEmptyConvolvedDims is not set, we further checks whether the op contains convolved dims.

Definition at line 847 of file LinalgInterfaces.cpp.

References getPreservedDims(), and inferConvolutionDimsImpl().

Referenced by mlir::linalg::isaConvolutionOpInterface(), and verifyConvolutionInterface().

◆ verifyContractionInterface()

LogicalResult mlir::linalg::detail::verifyContractionInterface ( Operation * op )

Verify that op conforms to ContractionOpInterface.

Verify that a LinalgOp op is a contraction.

A Linalg contraction is defined in general terms:

Has 2 input and 1 output shapes.
Has at least one reduction dimension.
Has only projected permutation indexing maps.
its body computes u5(u1(c) + u2(u3(a) * u4(b))) on some field (AddOpType, MulOpType), where u1, u2, u3, u4 and u5 represent scalar unary operations that may change the type (e.g. for mixed-precision). As a consequence, when vectorization of such an op occurs, the only special behavior is that the (unique) MulOpType is vectorized into a vector.contract. All other ops are handled in a generic fashion. In the future, we may wish to allow more input arguments and elementwise and constant operations that do not involve the reduction dimension(s).

Definition at line 550 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), getMatchContractionMessage(), and isContractionInterfaceImpl().

◆ verifyConvolutionInterface()

LogicalResult mlir::linalg::detail::verifyConvolutionInterface ( Operation * op )

Verify that op conforms to the ConvolutionOpInterface.

Definition at line 1012 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), getMatchConvolutionMessage(), and isConvolutionInterfaceImpl().

◆ verifyFillInterface()

LogicalResult mlir::linalg::detail::verifyFillInterface ( Operation * op )

Verify that op conforms to the FillOpInterface.

Definition at line 1044 of file LinalgInterfaces.cpp.

References mlir::Operation::emitError(), isFillInterfaceImpl(), NotLinalgOp, NotScalarInput, and WrongNumOperands.

◆ verifyStructuredOpInterface()

LogicalResult mlir::linalg::detail::verifyStructuredOpInterface ( Operation * op )

Verify that op conforms to the invariants of StructuredOpInterface.

Definition at line 1205 of file LinalgInterfaces.cpp.

References mlir::AffineMap::compose(), mlir::Operation::emitOpError(), mlir::Block::front(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::Block::getArgument(), mlir::getElementTypeOrSelf(), mlir::Block::getNumArguments(), mlir::AffineMap::getNumDims(), mlir::Operation::getNumOperands(), mlir::AffineMap::getNumResults(), mlir::AffineMap::getNumSymbols(), mlir::OpOperand::getOperandNumber(), mlir::AffineMap::getResult(), mlir::Value::getType(), max(), and min().

Classes

Enumerations

Functions