|
MLIR
18.0.0git
|
Namespaces | |
| detail | |
| ir_detail | |
Classes | |
| class | StorageLayout |
| Provides methods to access fields of a sparse tensor with the given encoding. More... | |
| class | SparseTensorType |
A wrapper around RankedTensorType, which has three goals: More... | |
| struct | SparsifierOptions |
| Options for the "sparsifier" pipeline. More... | |
| struct | TensorExp |
| Tensor expression. Represents an MLIR expression in tensor index notation. More... | |
| struct | LatPoint |
| Lattice point. More... | |
| class | Merger |
| A class to handle all iteration lattice operations. More... | |
| struct | Element |
| An element of a sparse tensor in coordinate-scheme representation (i.e., a pair of coordinates and value). More... | |
| struct | ElementLT |
Closure object for operator< on Element with a given rank. More... | |
| class | SparseTensorCOO |
A memory-resident sparse tensor in coordinate-scheme representation (a collection of Elements). More... | |
| class | SparseTensorReader |
| This class abstracts over the information stored in file headers, as well as providing the buffers and methods for parsing those headers. More... | |
| class | MapRef |
| A class for capturing the sparse tensor type map with a compact encoding. More... | |
| class | SparseTensorStorageBase |
Abstract base class for SparseTensorStorage<P,C,V>. More... | |
| class | SparseTensorStorage |
| A memory-resident sparse tensor using a storage scheme based on per-level sparse/dense annotations. More... | |
| class | CodegenEnv |
| The code generation environment class aggregates a number of data structures that are needed during the code generation phase of sparsification. More... | |
| class | FuncCallOrInlineGenerator |
| A helper class to simplify lowering operations with/without function calls. More... | |
| class | IterationGraphSorter |
| class | LoopEmitter |
| class | SparseTensorSpecifier |
| class | SparseTensorDescriptorImpl |
| A helper class around an array of values that corresponds to a sparse tensor. More... | |
| class | SparseTensorDescriptor |
| Uses ValueRange for immutable descriptors. More... | |
| class | MutSparseTensorDescriptor |
| Using SmallVector for mutable descriptor allows users to reuse it as a tmp buffers to append value for some special cases, though users should be responsible to restore the buffer to legal states after their use. More... | |
| class | SparsificationAndBufferizationPass |
| A pass that lowers tensor ops to memref ops, regardless of whether they are dense or sparse. More... | |
Typedefs | |
| using | index_type = uint64_t |
| This type is used in the public API at all places where MLIR expects values with the built-in type "index". More... | |
| using | complex64 = std::complex< double > |
| using | complex32 = std::complex< float > |
| using | Dimension = uint64_t |
| The type of dimension identifiers and dimension-ranks. More... | |
| using | Level = uint64_t |
| The type of level identifiers and level-ranks. More... | |
| using | Size = int64_t |
The type for individual components of a compile-time shape, including the value ShapedType::kDynamic (for shapes). More... | |
| using | FieldIndex = unsigned |
| The type of field indices. More... | |
| using | TensorId = unsigned |
Tensor identifiers, chosen to be the BlockArgument::getArgNumber of the value passed to Merger::buildTensorExp. More... | |
| using | LoopId = unsigned |
| Loop identifiers. More... | |
| using | TensorLoopId = unsigned |
A compressed representation of std::pair<TensorId, LoopId>. More... | |
| using | ExprId = unsigned |
TensorExp identifiers. More... | |
| using | LatPointId = unsigned |
LatPoint identifiers. More... | |
| using | LatSetId = unsigned |
LatSet identifiers. More... | |
| using | LvlLTPair = std::pair< Level, LevelType > |
| A pair of level and its corresponding LevelType of a tensor. More... | |
| using | LoopCoeffPair = std::pair< LoopId, unsigned > |
| A pair of loop id and its coefficients. More... | |
| using | LoopOrd = unsigned |
The position of a loop in the loop-stack, or the position of a LoopId in a topologically-sorted list of LoopIds. More... | |
| using | TensorLevel = unsigned |
Enumerations | |
| enum class | OverheadType : uint32_t { kIndex = 0 , kU64 = 1 , kU32 = 2 , kU16 = 3 , kU8 = 4 } |
| Encoding of overhead types (both position overhead and coordinate overhead), for "overloading" @newSparseTensor. More... | |
| enum class | PrimaryType : uint32_t { kF64 = 1 , kF32 = 2 , kF16 = 3 , kBF16 = 4 , kI64 = 5 , kI32 = 6 , kI16 = 7 , kI8 = 8 , kC64 = 9 , kC32 = 10 } |
| Encoding of the elemental type, for "overloading" @newSparseTensor. More... | |
| enum class | Action : uint32_t { kEmpty = 0 , kEmptyForward = 1 , kFromCOO = 2 , kFromReader = 4 , kToCOO = 5 , kPack = 7 , kSortCOOInPlace = 8 } |
| The actions performed by @newSparseTensor. More... | |
| enum class | LevelType : uint8_t { Undef = 0 , Dense = 4 , Compressed = 8 , CompressedNu = 9 , CompressedNo = 10 , CompressedNuNo = 11 , Singleton = 16 , SingletonNu = 17 , SingletonNo = 18 , SingletonNuNo = 19 , LooseCompressed = 32 , LooseCompressedNu = 33 , LooseCompressedNo = 34 , LooseCompressedNuNo = 35 , TwoOutOfFour = 64 } |
| This enum defines all the sparse representations supportable by the SparseTensor dialect. More... | |
| enum class | LevelFormat : uint8_t { Dense = 4 , Compressed = 8 , Singleton = 16 , LooseCompressed = 32 , TwoOutOfFour = 64 } |
| This enum defines all supported storage format without the level properties. More... | |
| enum class | LevelPropertyNondefault : uint8_t { Nonunique = 1 , Nonordered = 2 } |
| This enum defines all the nondefault properties for storage formats. More... | |
| enum class | SparseTensorFieldKind : uint32_t { StorageSpec = 0 , PosMemRef = static_cast<uint32_t>(StorageSpecifierKind::PosMemSize) , CrdMemRef = static_cast<uint32_t>(StorageSpecifierKind::CrdMemSize) , ValMemRef = static_cast<uint32_t>(StorageSpecifierKind::ValMemSize) } |
| ===-------------------------------------------------------------------—===// The sparse tensor storage scheme for a tensor is organized as a single compound type with the following fields. More... | |
| enum class | EmitCInterface : bool { Off = false , On = true } |
Shorthand aliases for the emitCInterface argument to getFunc(), createFuncCall(), and replaceOpWithFuncCall(). More... | |
| enum class | SortMask : unsigned { kIncludeDenseOutput = 0x1 , kIncludeDenseInput = 0x2 , kIncludeAll = 0x7 , kIncludeDense = 0x3 , kSparseOnly = 0x0 } |
| Iteration graph sorting. More... | |
| enum class | ExpArity { kNullary , kUnary , kBinary } |
Functions | |
| constexpr bool | isFloatingPrimaryType (PrimaryType valTy) |
| constexpr bool | isIntegralPrimaryType (PrimaryType valTy) |
| constexpr bool | isRealPrimaryType (PrimaryType valTy) |
| constexpr bool | isComplexPrimaryType (PrimaryType valTy) |
| constexpr const char * | toMLIRString (LevelType lt) |
| Returns string representation of the given dimension level type. More... | |
| constexpr bool | isValidLT (LevelType lt) |
Check that the LevelType contains a valid (possibly undefined) value. More... | |
| constexpr bool | isUndefLT (LevelType lt) |
Check if the LevelType is the special undefined value. More... | |
| constexpr bool | isDenseLT (LevelType lt) |
Check if the LevelType is dense (regardless of properties). More... | |
| constexpr bool | isCompressedLT (LevelType lt) |
Check if the LevelType is compressed (regardless of properties). More... | |
| constexpr bool | isSingletonLT (LevelType lt) |
Check if the LevelType is singleton (regardless of properties). More... | |
| constexpr bool | isLooseCompressedLT (LevelType lt) |
Check if the LevelType is loose compressed (regardless of properties). More... | |
| constexpr bool | is2OutOf4LT (LevelType lt) |
Check if the LevelType is 2OutOf4 (regardless of properties). More... | |
| constexpr bool | isWithPosLT (LevelType lt) |
Check if the LevelType needs positions array. More... | |
| constexpr bool | isWithCrdLT (LevelType lt) |
Check if the LevelType needs coordinates array. More... | |
| constexpr bool | isOrderedLT (LevelType lt) |
Check if the LevelType is ordered (regardless of storage format). More... | |
| constexpr bool | isUniqueLT (LevelType lt) |
Check if the LevelType is unique (regardless of storage format). More... | |
| constexpr std::optional< LevelFormat > | getLevelFormat (LevelType lt) |
| Convert a LevelType to its corresponding LevelFormat. More... | |
| constexpr std::optional< LevelType > | buildLevelType (LevelFormat lf, bool ordered, bool unique) |
| Convert a LevelFormat to its corresponding LevelType with the given properties. More... | |
| constexpr uint64_t | encodeDim (uint64_t i, uint64_t cf, uint64_t cm) |
| Bit manipulations for affine encoding. More... | |
| constexpr uint64_t | encodeLvl (uint64_t i, uint64_t c, uint64_t ii) |
| constexpr bool | isEncodedFloor (uint64_t v) |
| constexpr bool | isEncodedMod (uint64_t v) |
| constexpr bool | isEncodedMul (uint64_t v) |
| constexpr uint64_t | decodeIndex (uint64_t v) |
| constexpr uint64_t | decodeConst (uint64_t v) |
| constexpr uint64_t | decodeMulc (uint64_t v) |
| constexpr uint64_t | decodeMuli (uint64_t v) |
| template<typename T > | |
| RankedTensorType | getRankedTensorType (T &&t) |
Convenience method to abbreviate casting getType(). More... | |
| template<typename T > | |
| MemRefType | getMemRefType (T &&t) |
Convenience method to abbreviate casting getType(). More... | |
| SparseTensorEncodingAttr | getSparseTensorEncoding (Type type) |
| Convenience method to get a sparse encoding attribute from a type. More... | |
| bool | hasAnySparseOperand (Operation *op) |
| Returns true iff MLIR operand has any sparse operand. More... | |
| bool | hasAnySparseResult (Operation *op) |
| Returns true iff MLIR operand has any sparse result. More... | |
| bool | hasAnySparseOperandOrResult (Operation *op) |
| Returns true iff MLIR operand has any sparse operand or result. More... | |
| bool | hasAnyNonIdentityOperandsOrResults (Operation *op) |
| Returns true iff MLIR operation has any sparse tensor with non-identity dim2lvl maps. More... | |
| AffineMap | inferLvlToDim (AffineMap dimToLvl, MLIRContext *context) |
| Given the dimToLvl map, infers the lvlToDim map, or returns empty Affine map when inference fails. More... | |
| AffineMap | inverseBlockSparsity (AffineMap dimToLvl, MLIRContext *context) |
| Returns the lvlToDim map for the given dimToLvl map specific to the block sparse cases. More... | |
| SmallVector< unsigned > | getBlockSize (AffineMap dimToLvl) |
| Given the dimToLvl map, returns the block sizes in a vector. More... | |
| bool | isBlockSparsity (AffineMap dimToLvl) |
| Given the dimToLvl map, returns if it's block sparsity. More... | |
| Dimension | toDim (SparseTensorEncodingAttr enc, Level l) |
| Convenience method to translate the given level to the corresponding dimension. More... | |
| Level | toLvl (SparseTensorEncodingAttr enc, Dimension d) |
| Convenience method to translate the given dimension to the corresponding level. More... | |
| StorageSpecifierKind | toSpecifierKind (SparseTensorFieldKind kind) |
| SparseTensorFieldKind | toFieldKind (StorageSpecifierKind kind) |
| unsigned | getNumFieldsFromEncoding (SparseTensorEncodingAttr enc) |
| unsigned | getNumDataFieldsFromEncoding (SparseTensorEncodingAttr enc) |
| void | foreachFieldInSparseTensor (SparseTensorEncodingAttr enc, llvm::function_ref< bool(FieldIndex, SparseTensorFieldKind, Level, LevelType)> callback) |
| void | foreachFieldAndTypeInSparseTensor (SparseTensorType, llvm::function_ref< bool(Type, FieldIndex, SparseTensorFieldKind, Level, LevelType)>) |
| SparseTensorType | getSparseTensorType (Value val) |
| Convenience methods to obtain a SparseTensorType from a Value. More... | |
| std::optional< SparseTensorType > | tryGetSparseTensorType (Value val) |
| void | buildSparsifier (OpPassManager &pm, const SparsifierOptions &options) |
Adds the "sparsifier" pipeline to the OpPassManager. More... | |
| void | registerSparseTensorPipelines () |
Registers all pipelines for the sparse_tensor dialect. More... | |
| void | registerTransformDialectExtension (DialectRegistry ®istry) |
| void | registerBufferizableOpInterfaceExternalModels (DialectRegistry ®istry) |
| OverheadType | overheadTypeEncoding (unsigned width) |
| Converts an overhead storage bitwidth to its internal type-encoding. More... | |
| OverheadType | overheadTypeEncoding (Type tp) |
| Converts an overhead storage type to its internal type-encoding. More... | |
| Type | getOverheadType (Builder &builder, OverheadType ot) |
| Converts the internal type-encoding for overhead storage to an mlir::Type. More... | |
| OverheadType | posTypeEncoding (SparseTensorEncodingAttr enc) |
| Returns the OverheadType for position overhead storage. More... | |
| OverheadType | crdTypeEncoding (SparseTensorEncodingAttr enc) |
| Returns the OverheadType for coordinate overhead storage. More... | |
| StringRef | overheadTypeFunctionSuffix (OverheadType ot) |
| Convert OverheadType to its function-name suffix. More... | |
| StringRef | overheadTypeFunctionSuffix (Type overheadTp) |
| Converts an overhead storage type to its function-name suffix. More... | |
| PrimaryType | primaryTypeEncoding (Type elemTp) |
| Converts a primary storage type to its internal type-encoding. More... | |
| StringRef | primaryTypeFunctionSuffix (PrimaryType pt) |
| Convert PrimaryType to its function-name suffix. More... | |
| StringRef | primaryTypeFunctionSuffix (Type elemTp) |
| Converts a primary storage type to its function-name suffix. More... | |
| Value | genCast (OpBuilder &builder, Location loc, Value value, Type dstTy) |
| Add type casting between arith and index types when needed. More... | |
| Value | genScalarToTensor (OpBuilder &builder, Location loc, Value elem, Type dstTp) |
| Add conversion from scalar to given type (possibly a 0-rank tensor). More... | |
| Value | genIndexLoad (OpBuilder &builder, Location loc, Value mem, Value s) |
| Generates a pointer/index load from the sparse storage scheme. More... | |
| TypedAttr | getOneAttr (Builder &builder, Type tp) |
| Generates a 1-valued attribute of the given type. More... | |
| Value | genIsNonzero (OpBuilder &builder, Location loc, Value v) |
Generates the comparison v != 0 where v is of numeric type. More... | |
| void | genReshapeDstShape (OpBuilder &builder, Location loc, SmallVectorImpl< Value > &dstShape, ArrayRef< Value > srcShape, ArrayRef< Size > staticDstShape, ArrayRef< ReassociationIndices > reassociation) |
| Computes the shape of destination tensor of a reshape operator. More... | |
| void | reshapeCvs (OpBuilder &builder, Location loc, ArrayRef< ReassociationIndices > reassociation, ValueRange srcSizes, ValueRange srcCvs, ValueRange dstSizes, SmallVectorImpl< Value > &dstCvs) |
| Reshape coordinates during a reshaping operation. More... | |
| FlatSymbolRefAttr | getFunc (ModuleOp module, StringRef name, TypeRange resultType, ValueRange operands, EmitCInterface emitCInterface) |
| Returns a function reference (first hit also inserts into module). More... | |
| func::CallOp | createFuncCall (OpBuilder &builder, Location loc, StringRef name, TypeRange resultType, ValueRange operands, EmitCInterface emitCInterface) |
Creates a CallOp to the function reference returned by getFunc() in the builder's module. More... | |
| Type | getOpaquePointerType (MLIRContext *ctx) |
Returns the equivalent of void* for opaque arguments to the execution engine. More... | |
| Type | getOpaquePointerType (Builder &builder) |
| Value | genAlloca (OpBuilder &builder, Location loc, Value sz, Type tp) |
Generates an uninitialized temporary buffer of the given size and type, but returns it as type memref<? x $tp> (rather than as type memref<$sz x $tp>). More... | |
| Value | genAlloca (OpBuilder &builder, Location loc, unsigned sz, Type tp, bool staticShape=false) |
Generates an uninitialized temporary buffer of the given size and type, and returns it as type memref<? x $tp> (staticShape=false) or memref<$sz x $tp> (staticShape=true). More... | |
| Value | genAllocaScalar (OpBuilder &builder, Location loc, Type tp) |
Generates an uninitialized temporary buffer with room for one value of the given type, and returns the memref<$tp>. More... | |
| Value | allocaBuffer (OpBuilder &builder, Location loc, ValueRange values) |
Generates a temporary buffer, initializes it with the given contents, and returns it as type memref<? x $tp> (rather than specifying the size of the buffer). More... | |
| Value | allocDenseTensor (OpBuilder &builder, Location loc, RankedTensorType tensorTp, ValueRange sizes) |
| Generates code to allocate a buffer of the given type, and zero initialize it. More... | |
| void | deallocDenseTensor (OpBuilder &builder, Location loc, Value buffer) |
| Generates code to deallocate a dense buffer. More... | |
| void | sizesFromSrc (OpBuilder &builder, SmallVectorImpl< Value > &sizes, Location loc, Value src) |
| Populates given sizes array from dense tensor or sparse tensor constant. More... | |
| MemRefType | get1DMemRefType (Type etp, bool withLayout) |
| Generates a 1D MemRefType with a dynamic size. More... | |
| Operation * | getTop (Operation *op) |
| Scans to top of generated loop. More... | |
| void | foreachInSparseConstant (OpBuilder &builder, Location loc, SparseElementsAttr attr, AffineMap order, function_ref< void(ArrayRef< Value >, Value)> callback) |
| Iterate over a sparse constant, generates constantOp for value and coordinates. More... | |
| SmallVector< Value > | loadAll (OpBuilder &builder, Location loc, size_t size, Value mem, size_t offsetIdx=0, Value offsetVal=Value()) |
Loads size-many values from the memref, which must have rank-1 and size greater-or-equal to size. More... | |
| void | storeAll (OpBuilder &builder, Location loc, Value mem, ValueRange vs, size_t offsetIdx=0, Value offsetVal=Value()) |
Stores all the values of vs into the memref mem, which must have rank-1 and size greater-or-equal to vs.size(). More... | |
| TypedValue< BaseMemRefType > | genToMemref (OpBuilder &builder, Location loc, Value tensor) |
| Value | genToPositions (OpBuilder &builder, Location loc, Value tensor, Level lvl) |
Infers the result type and generates ToPositionsOp. More... | |
| Value | genToCoordinates (OpBuilder &builder, Location loc, Value tensor, Level lvl, Level cooStart) |
Infers the result type and generates ToCoordinatesOp. More... | |
| Value | genToCoordinatesBuffer (OpBuilder &builder, Location loc, Value tensor) |
Infers the result type and generates ToCoordinatesBufferOp. More... | |
| Value | genToValues (OpBuilder &builder, Location loc, Value tensor) |
Infers the result type and generates ToValuesOp. More... | |
| Value | genValMemSize (OpBuilder &builder, Location loc, Value tensor) |
| Generates code to retrieve the values size for the sparse tensor. More... | |
| Value | createOrFoldSliceOffsetOp (OpBuilder &builder, Location loc, Value tensor, Dimension dim) |
| Generates code to retrieve the slice offset for the sparse tensor slice, return a constant if the offset is statically known. More... | |
| Value | createOrFoldSliceStrideOp (OpBuilder &builder, Location loc, Value tensor, Dimension dim) |
| Generates code to retrieve the slice slice for the sparse tensor slice, return a constant if the offset is statically known. More... | |
| Value | genReader (OpBuilder &builder, Location loc, SparseTensorType stt, Value tensor, SmallVectorImpl< Value > &dimSizesValues, Value &dimSizesBuffer) |
| Generates code that opens a reader and sets the dimension sizes. More... | |
| Value | genMapBuffers (OpBuilder &builder, Location loc, SparseTensorType stt, ArrayRef< Value > dimSizesValues, Value dimSizesBuffer, SmallVectorImpl< Value > &lvlSizesValues, Value &dim2lvlBuffer, Value &lvl2dimBuffer) |
| Generates code to set up the buffer parameters for a map. More... | |
| Value | constantZero (OpBuilder &builder, Location loc, Type tp) |
| Generates a 0-valued constant of the given type. More... | |
| Value | constantOne (OpBuilder &builder, Location loc, Type tp) |
| Generates a 1-valued constant of the given type. More... | |
| Value | constantIndex (OpBuilder &builder, Location loc, int64_t i) |
Generates a constant of index type. More... | |
| Value | constantI64 (OpBuilder &builder, Location loc, int64_t i) |
Generates a constant of i64 type. More... | |
| Value | constantI32 (OpBuilder &builder, Location loc, int32_t i) |
Generates a constant of i32 type. More... | |
| Value | constantI16 (OpBuilder &builder, Location loc, int16_t i) |
Generates a constant of i16 type. More... | |
| Value | constantI8 (OpBuilder &builder, Location loc, int8_t i) |
Generates a constant of i8 type. More... | |
| Value | constantI1 (OpBuilder &builder, Location loc, bool b) |
Generates a constant of i1 type. More... | |
| Value | constantAction (OpBuilder &builder, Location loc, Action action) |
Generates a constant of the given Action. More... | |
| Value | constantOverheadTypeEncoding (OpBuilder &builder, Location loc, unsigned width) |
| Generates a constant of the internal type-encoding for overhead storage. More... | |
| Value | constantPosTypeEncoding (OpBuilder &builder, Location loc, SparseTensorEncodingAttr enc) |
| Generates a constant of the internal type-encoding for position overhead storage. More... | |
| Value | constantCrdTypeEncoding (OpBuilder &builder, Location loc, SparseTensorEncodingAttr enc) |
| Generates a constant of the internal type-encoding for coordinate overhead storage. More... | |
| Value | constantPrimaryTypeEncoding (OpBuilder &builder, Location loc, Type elemTp) |
| Generates a constant of the internal type-encoding for primary storage. More... | |
| Value | constantLevelTypeEncoding (OpBuilder &builder, Location loc, LevelType lt) |
| Generates a constant of the internal dimension level type encoding. More... | |
| bool | isZeroRankedTensorOrScalar (Type type) |
| UnrealizedConversionCastOp | getTuple (Value tensor) |
| Returns the "tuple" value of the adapted tensor. More... | |
| Value | genTuple (OpBuilder &builder, Location loc, Type tp, ValueRange values) |
| Packs the given values as a "tuple" value. More... | |
| Value | genTuple (OpBuilder &builder, Location loc, SparseTensorDescriptor desc) |
| SparseTensorDescriptor | getDescriptorFromTensorTuple (Value tensor) |
| MutSparseTensorDescriptor | getMutDescriptorFromTensorTuple (Value tensor, SmallVectorImpl< Value > &fields) |
| static bool | containsSparseTensor (TypeRange types) |
Return true if one of the given types is a sparse tensor type. More... | |
| static ExpArity | getExpArity (TensorExp::Kind k) |
| static const char * | kindToOpSymbol (TensorExp::Kind kind) |
| static bool | isAdmissibleBranchExp (Operation *op, Block *block, Value v) |
| Ensures that the sparsifier can generate code for expression. More... | |
| static bool | isAdmissibleBranch (Operation *op, Region ®ion) |
| Ensures that the sparsifier can generate code for branch. More... | |
| static Value | insertYieldOp (RewriterBase &rewriter, Location loc, Region ®ion, ValueRange vals) |
| static Value | buildUnaryPresent (RewriterBase &rewriter, Location loc, Operation *op, Value v0) |
| static Value | buildBinaryOverlap (RewriterBase &rewriter, Location loc, Operation *op, Value v0, Value v1) |
| using mlir::sparse_tensor::complex32 = typedef std::complex<float> |
| using mlir::sparse_tensor::complex64 = typedef std::complex<double> |
| using mlir::sparse_tensor::Dimension = typedef uint64_t |
The type of dimension identifiers and dimension-ranks.
Definition at line 35 of file SparseTensor.h.
| using mlir::sparse_tensor::ExprId = typedef unsigned |
TensorExp identifiers.
These are allocated by Merger::addExp, and serve as unique identifiers for the corresponding TensorExp object.
| using mlir::sparse_tensor::FieldIndex = typedef unsigned |
The type of field indices.
This alias is to help code be more self-documenting; unfortunately it is not type-checked, so it only provides documentation rather than doing anything to prevent mixups.
Definition at line 107 of file SparseTensorStorageLayout.h.
| using mlir::sparse_tensor::index_type = typedef uint64_t |
This type is used in the public API at all places where MLIR expects values with the built-in type "index".
For now, we simply assume that type is 64-bit, but targets with different "index" bitwidths should link with an alternatively built runtime support library.
| using mlir::sparse_tensor::LatPointId = typedef unsigned |
LatPoint identifiers.
These are allocated by Merger::addLat, and serve as unique identifiers for the corresponding LatPoint object.
| using mlir::sparse_tensor::LatSetId = typedef unsigned |
LatSet identifiers.
These are allocated by Merger::addSet (and by other methods calling that one), and serve as unique identifiers for the corresponding SmallVector<LatPointId> object.
| using mlir::sparse_tensor::Level = typedef uint64_t |
The type of level identifiers and level-ranks.
Definition at line 38 of file SparseTensor.h.
| using mlir::sparse_tensor::LoopCoeffPair = typedef std::pair<LoopId, unsigned> |
| using mlir::sparse_tensor::LoopId = typedef unsigned |
| using mlir::sparse_tensor::LoopOrd = typedef unsigned |
The position of a loop in the loop-stack, or the position of a LoopId in a topologically-sorted list of LoopIds.
Although this type may have the same cardinality as LoopId, it must not be confused with that type. The LoopId type is used by the Merger as a unique identifier for loop-variables, regardless of the ordering of those loops. Whereas the LoopOrd type is used by the LoopEmitter (and CodegenEnv) to refer to the actual order in which loops are generated.
TODO: further explicate the correspondences between these various types. In particular, since the $dim argument to linalg::IndexOp is a De Bruijn index, it seems like that should correspond to LoopOrd, and yet the Merger has that correspond with LoopId instead. In addition LoopEmitter::genAffine has AffineDimExpr::position correspond to LoopId, however it is unclear what the providence of those AffineDimExpr is.
Definition at line 43 of file LoopEmitter.h.
| using mlir::sparse_tensor::LvlLTPair = typedef std::pair<Level, LevelType> |
| using mlir::sparse_tensor::Size = typedef int64_t |
The type for individual components of a compile-time shape, including the value ShapedType::kDynamic (for shapes).
Definition at line 42 of file SparseTensor.h.
| using mlir::sparse_tensor::TensorId = typedef unsigned |
Tensor identifiers, chosen to be the BlockArgument::getArgNumber of the value passed to Merger::buildTensorExp.
| using mlir::sparse_tensor::TensorLevel = typedef unsigned |
Definition at line 46 of file LoopEmitter.h.
| using mlir::sparse_tensor::TensorLoopId = typedef unsigned |
|
strong |
|
strong |
Shorthand aliases for the emitCInterface argument to getFunc(), createFuncCall(), and replaceOpWithFuncCall().
| Enumerator | |
|---|---|
| Off | |
| On | |
Definition at line 36 of file CodegenUtils.h.
|
strong |
| Enumerator | |
|---|---|
| kNullary | |
| kUnary | |
| kBinary | |
Definition at line 22 of file Merger.cpp.
|
strong |
|
strong |
|
strong |
This enum defines all the sparse representations supportable by the SparseTensor dialect.
We use a lightweight encoding to encode both the "format" per se (dense, compressed, singleton, loose_compressed, two-out-of-four) as well as the "properties" (ordered, unique). The encoding is chosen for performance of the runtime library, and thus may change in future versions; consequently, client code should use the predicate functions defined below, rather than relying on knowledge about the particular binary encoding.
The Undef "format" is a special value used internally for cases where we need to store an undefined or indeterminate LevelType. It should not be used externally, since it does not indicate an actual/representable format.
|
strong |
|
strong |
|
strong |
Iteration graph sorting.
| Enumerator | |
|---|---|
| kIncludeDenseOutput | |
| kIncludeDenseInput | |
| kIncludeAll | |
| kIncludeDense | |
| kSparseOnly | |
Definition at line 25 of file IterationGraphSorter.h.
|
strong |
===-------------------------------------------------------------------—===// The sparse tensor storage scheme for a tensor is organized as a single compound type with the following fields.
Note that every memref with ? size actually behaves as a "vector", i.e. the stored size is the capacity and the used size resides in the storage_specifier struct.
struct { ; per-level l: ; if dense: <nothing> ; if compressed: memref<? x pos> positions ; positions for level l memref<? x crd> coordinates ; coordinates for level l ; if loose-compressed: memref<? x pos> positions ; lo/hi position pairs for level l memref<? x crd> coordinates ; coordinates for level l ; if singleton/2-out-of-4: memref<? x crd> coordinates ; coordinates for level l
memref<? x eltType> values ; values
struct sparse_tensor.storage_specifier { array<rank x int> lvlSizes ; sizes/cardinalities for each level array<n x int> memSizes; ; sizes/lengths for each data memref } };
In addition, for a "trailing COO region", defined as a compressed level followed by one or more singleton levels, the default SOA storage that is inherent to the TACO format is optimized into an AOS storage where all coordinates of a stored element appear consecutively. In such cases, a special operation (sparse_tensor.coordinates_buffer) must be used to access the AOS coordinates array. In the code below, the method getCOOStart is used to find the start of the "trailing COO region".
If the sparse tensor is a slice (produced by tensor.extract_slice operation), instead of allocating a new sparse tensor for it, it reuses the same sets of MemRefs but attaching a additional set of slicing-metadata for per-dimension slice offset and stride.
Examples.
#CSR storage of 2-dim matrix yields memref<?xindex> ; positions-1 memref<?xindex> ; coordinates-1 memref<?xf64> ; values struct<(array<2 x i64>, array<3 x i64>)>) ; lvl0, lvl1, 3xsizes
#COO storage of 2-dim matrix yields memref<?xindex>, ; positions-0, essentially [0,sz] memref<?xindex> ; AOS coordinates storage memref<?xf64> ; values struct<(array<2 x i64>, array<3 x i64>)>) ; lvl0, lvl1, 3xsizes
Slice on #COO storage of 2-dim matrix yields ;; Inherited from the original sparse tensors memref<?xindex>, ; positions-0, essentially [0,sz] memref<?xindex> ; AOS coordinates storage memref<?xf64> ; values struct<(array<2 x i64>, array<3 x i64>, ; lvl0, lvl1, 3xsizes ;; Extra slicing-metadata array<2 x i64>, array<2 x i64>)>) ; dim offset, dim stride.
===-------------------------------------------------------------------—===//
| Enumerator | |
|---|---|
| StorageSpec | |
| PosMemRef | |
| CrdMemRef | |
| ValMemRef | |
Definition at line 87 of file SparseTensorStorageLayout.h.
| Value mlir::sparse_tensor::allocaBuffer | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ValueRange | values | ||
| ) |
Generates a temporary buffer, initializes it with the given contents, and returns it as type memref<? x $tp> (rather than specifying the size of the buffer).
Definition at line 380 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), and genAlloca().
Referenced by genMapBuffers(), and genReader().
| Value mlir::sparse_tensor::allocDenseTensor | ( | OpBuilder & | builder, |
| Location | loc, | ||
| RankedTensorType | tensorTp, | ||
| ValueRange | sizes | ||
| ) |
Generates code to allocate a buffer of the given type, and zero initialize it.
If the buffer type has any dynamic sizes, then the sizes parameter should be as filled by sizesFromPtr(); that way we can reuse the genDimSizeCall() results generated by sizesFromPtr().
Definition at line 392 of file CodegenUtils.cpp.
References constantZero(), mlir::OpBuilder::create(), and mlir::get().
|
static |
Definition at line 1524 of file Merger.cpp.
Referenced by mlir::sparse_tensor::Merger::buildExp().
|
constexpr |
Convert a LevelFormat to its corresponding LevelType with the given properties.
Returns std::nullopt when the properties are not applicable for the input level format.
Definition at line 314 of file Enums.h.
References isValidLT().
Referenced by mlir::sparse_tensor::SparseTensorType::getCOOType(), and getNormalizedEncodingForSpecifier().
| void mlir::sparse_tensor::buildSparsifier | ( | OpPassManager & | pm, |
| const SparsifierOptions & | options | ||
| ) |
Adds the "sparsifier" pipeline to the OpPassManager.
This is the standard pipeline for taking sparsity-agnostic IR using the sparse-tensor type and lowering it to LLVM IR with concrete representations and algorithms for sparse tensors.
Definition at line 32 of file SparseTensorPipelines.cpp.
References mlir::OpPassManager::addNestedPass(), mlir::OpPassManager::addPass(), mlir::createCanonicalizerPass(), mlir::createConvertComplexToLibmPass(), mlir::createConvertComplexToStandardPass(), mlir::createConvertLinalgToLoopsPass(), mlir::createConvertMathToLibmPass(), mlir::createConvertSCFToCFPass(), mlir::createConvertVectorToSCFPass(), mlir::memref::createExpandReallocPass(), mlir::memref::createExpandStridedMetadataPass(), mlir::bufferization::createFinalizingBufferizePass(), mlir::createLinalgGeneralizationPass(), mlir::createLowerAffinePass(), mlir::createReconcileUnrealizedCastsPass(), mlir::createSparseGPUCodegenPass(), mlir::createSparsificationAndBufferizationPass(), mlir::createStorageSpecifierToLLVMPass(), mlir::createStripDebugInfoPass(), mlir::getBufferizationOptionsForSparsification(), and options.
Referenced by registerSparseTensorPipelines().
|
static |
Definition at line 1510 of file Merger.cpp.
Referenced by mlir::sparse_tensor::Merger::buildExp().
|
inline |
Generates a constant of the given Action.
Definition at line 391 of file CodegenUtils.h.
References constantI32().
|
inline |
Generates a constant of the internal type-encoding for coordinate overhead storage.
Definition at line 411 of file CodegenUtils.h.
References constantOverheadTypeEncoding().
Generates a constant of i1 type.
Definition at line 386 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Referenced by createEqCompare(), createPartitionFunc(), endIf(), endLoop(), genCompressed(), genIf(), and genSparseReducedAffineCond().
Generates a constant of i16 type.
Definition at line 376 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Generates a constant of i32 type.
Definition at line 371 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Referenced by constantAction(), constantOverheadTypeEncoding(), and constantPrimaryTypeEncoding().
Generates a constant of i64 type.
Definition at line 366 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Referenced by createQuickSortFunc().
Generates a constant of i8 type.
Definition at line 381 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Referenced by constantLevelTypeEncoding().
Generates a constant of index type.
Definition at line 361 of file CodegenUtils.h.
References mlir::OpBuilder::create().
Referenced by allocaBuffer(), allocSchemeForRank(), concatSizesFromInputs(), createAllocFields(), createBinarySearchFunc(), createChoosePivot(), createDimSizes(), createHeapSortFunc(), createOrFoldSliceOffsetOp(), createOrFoldSliceStrideOp(), createPartitionFunc(), createQuickSort(), createQuickSortFunc(), createScanLoop(), createShiftDownFunc(), createSortStableFunc(), createSubTwoDividedByTwo(), forEachIJPairInXs(), genAlloca(), genCompressed(), genEndInsert(), genMapBuffers(), genReader(), genReshapeDstShape(), mlir::sparse_tensor::SparseTensorDescriptor::getCrdMemRefOrView(), loadAll(), parseGEPIndices(), reshapeCvs(), sizesForTensor(), and storeAll().
|
inline |
Generates a constant of the internal dimension level type encoding.
Definition at line 424 of file CodegenUtils.h.
References constantI8().
Generates a 1-valued constant of the given type.
This supports all the same types as constantZero.
Definition at line 350 of file CodegenUtils.h.
References mlir::OpBuilder::create(), mlir::Builder::getArrayAttr(), getOneAttr(), and mlir::Builder::getZeroAttr().
Referenced by createFor(), and createPartitionFunc().
|
inline |
Generates a constant of the internal type-encoding for overhead storage.
Definition at line 396 of file CodegenUtils.h.
References constantI32(), and overheadTypeEncoding().
Referenced by constantCrdTypeEncoding(), and constantPosTypeEncoding().
|
inline |
Generates a constant of the internal type-encoding for position overhead storage.
Definition at line 404 of file CodegenUtils.h.
References constantOverheadTypeEncoding().
|
inline |
Generates a constant of the internal type-encoding for primary storage.
Definition at line 417 of file CodegenUtils.h.
References constantI32(), and primaryTypeEncoding().
Referenced by genReader().
Generates a 0-valued constant of the given type.
In addition to the scalar types (ComplexType, FloatType, IndexType, IntegerType), this also works for RankedTensorType and VectorType (for which it generates a constant DenseElementsAttr of zeros).
Definition at line 339 of file CodegenUtils.h.
References mlir::OpBuilder::create(), mlir::Builder::getArrayAttr(), and mlir::Builder::getZeroAttr().
Referenced by allocDenseTensor(), allocSchemeForRank(), createAllocation(), createAllocFields(), createFor(), genEndInsert(), genIsNonzero(), and mlir::sparse_tensor::LoopEmitter::initializeLoopEmit().
|
static |
Return true if one of the given types is a sparse tensor type.
Definition at line 40 of file SparsificationAndBufferizationPass.cpp.
References getSparseTensorEncoding().
| OverheadType mlir::sparse_tensor::crdTypeEncoding | ( | SparseTensorEncodingAttr | enc | ) |
Returns the OverheadType for coordinate overhead storage.
Definition at line 76 of file CodegenUtils.cpp.
References overheadTypeEncoding().
| func::CallOp mlir::sparse_tensor::createFuncCall | ( | OpBuilder & | builder, |
| Location | loc, | ||
| StringRef | name, | ||
| TypeRange | resultType, | ||
| ValueRange | operands, | ||
| EmitCInterface | emitCInterface | ||
| ) |
Creates a CallOp to the function reference returned by getFunc() in the builder's module.
Definition at line 343 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), mlir::OpBuilder::getBlock(), getFunc(), mlir::Operation::getParentOfType(), and mlir::Block::getParentOp().
Referenced by dumpIndexMemRef(), and genReader().
| Value mlir::sparse_tensor::createOrFoldSliceOffsetOp | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor, | ||
| Dimension | dim | ||
| ) |
Generates code to retrieve the slice offset for the sparse tensor slice, return a constant if the offset is statically known.
Definition at line 596 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), getSparseTensorEncoding(), and mlir::Value::getType().
Referenced by genSliceOffset().
| Value mlir::sparse_tensor::createOrFoldSliceStrideOp | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor, | ||
| Dimension | dim | ||
| ) |
Generates code to retrieve the slice slice for the sparse tensor slice, return a constant if the offset is statically known.
Definition at line 606 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), getSparseTensorEncoding(), and mlir::Value::getType().
Referenced by genSliceStride().
Generates code to deallocate a dense buffer.
Definition at line 409 of file CodegenUtils.cpp.
References mlir::OpBuilder::create().
|
constexpr |
|
constexpr |
|
constexpr |
|
constexpr |
|
constexpr |
Bit manipulations for affine encoding.
Note that because the indices in the mappings refer to dimensions and levels (and not the sizes of these dimensions and levels), the 64-bit encoding gives ample room for a compact encoding of affine operations in the higher bits. Pure permutations still allow for 60-bit indices. But non-permutations reserve 20-bits for the potential three components (index i, constant, index ii).
The compact encoding is as follows:
0xffffffffffffffff |0000 | 60-bit idx| e.g. i |0001 floor| 20-bit const|20-bit idx| e.g. i floor c |0010 mod | 20-bit const|20-bit idx| e.g. i mod c |0011 mul |20-bit idx|20-bit const|20-bit idx| e.g. i + c * ii
This encoding provides sufficient generality for currently supported sparse tensor types. To generalize this more, we will need to provide a broader encoding scheme for affine functions. Also, the library encoding may be replaced with pure "direct-IR" code in the future.
Definition at line 531 of file Enums.h.
Referenced by genMapBuffers().
|
constexpr |
Definition at line 543 of file Enums.h.
Referenced by genMapBuffers().
| void mlir::sparse_tensor::foreachFieldAndTypeInSparseTensor | ( | SparseTensorType | stt, |
| llvm::function_ref< bool(Type, FieldIndex, SparseTensorFieldKind, Level, LevelType)> | callback | ||
| ) |
Definition at line 94 of file SparseTensorDialect.cpp.
References CrdMemRef, mlir::sparse_tensor::StorageLayout::foreachField(), mlir::get(), mlir::sparse_tensor::SparseTensorType::getCrdType(), mlir::sparse_tensor::SparseTensorType::getElementType(), mlir::sparse_tensor::SparseTensorType::getEncoding(), mlir::sparse_tensor::SparseTensorType::getPosType(), mlir::sparse_tensor::SparseTensorType::hasEncoding(), PosMemRef, StorageSpec, and ValMemRef.
Referenced by convertSparseTensorType(), and createAllocFields().
|
inline |
Definition at line 165 of file SparseTensorStorageLayout.h.
References mlir::sparse_tensor::StorageLayout::foreachField().
| void mlir::sparse_tensor::foreachInSparseConstant | ( | OpBuilder & | builder, |
| Location | loc, | ||
| SparseElementsAttr | attr, | ||
| AffineMap | order, | ||
| function_ref< void(ArrayRef< Value >, Value)> | callback | ||
| ) |
Iterate over a sparse constant, generates constantOp for value and coordinates.
E.g., sparse<[ [0], [28], [31] ], [ (-5.13, 2.0), (3.0, 4.0), (5.0, 6.0) ] > => c1 = arith.constant 0 v1 = complex.constant (5.13, 2.0) callback({c1}, v1)
c2 = arith.constant 28 v2 = complex.constant (3.0, 4.0) callback({c2}, v2)
c3 = arith.constant 31 v3 = complex.constant (5.0, 6.0) callback({c3}, v3)
Definition at line 432 of file CodegenUtils.cpp.
References mlir::AffineMap::compose(), mlir::Builder::getMultiDimIdentityMap(), mlir::AffineMap::getNumResults(), and getRankedTensorType().
| Value mlir::sparse_tensor::genAlloca | ( | OpBuilder & | builder, |
| Location | loc, | ||
| unsigned | sz, | ||
| Type | tp, | ||
| bool | staticShape = false |
||
| ) |
Generates an uninitialized temporary buffer of the given size and type, and returns it as type memref<? x $tp> (staticShape=false) or memref<$sz x $tp> (staticShape=true).
Definition at line 360 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), genAlloca(), and mlir::get().
Generates an uninitialized temporary buffer of the given size and type, but returns it as type memref<? x $tp> (rather than as type memref<$sz x $tp>).
Definition at line 369 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), and mlir::get().
Referenced by allocaBuffer(), allocSlicePosBuf(), and genAlloca().
Generates an uninitialized temporary buffer with room for one value of the given type, and returns the memref<$tp>.
Definition at line 375 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), and mlir::get().
Add type casting between arith and index types when needed.
Definition at line 151 of file CodegenUtils.cpp.
References mlir::convertScalarToDtype(), mlir::OpBuilder::create(), and mlir::Value::getType().
Referenced by genCompressed(), genLoad(), genScalarToTensor(), and genStore().
Generates a pointer/index load from the sparse storage scheme.
Narrower data types need to be zero extended before casting the value into the index type used for looping and indexing.
Definition at line 177 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), mlir::Builder::getI64Type(), mlir::Builder::getIndexType(), mlir::Type::getIntOrFloatBitWidth(), and mlir::Value::getType().
Referenced by mlir::sparse_tensor::LoopEmitter::enterFilterLoopOverTensorAtLvl(), genSparseReducedAffineCond(), loadSlicePos(), loadSlicePosPtr(), and loadSlicePosTupleNum().
| Value mlir::sparse_tensor::genIsNonzero | ( | OpBuilder & | builder, |
| mlir::Location | loc, | ||
| Value | v | ||
| ) |
Generates the comparison v != 0 where v is of numeric type.
For floating types, we use the "unordered" comparator (i.e., returns true if v is NaN).
Definition at line 204 of file CodegenUtils.cpp.
References constantZero(), mlir::OpBuilder::create(), mlir::Value::getType(), and mlir::Type::isIntOrIndex().
| Value mlir::sparse_tensor::genMapBuffers | ( | OpBuilder & | builder, |
| Location | loc, | ||
| SparseTensorType | stt, | ||
| ArrayRef< Value > | dimSizesValues, | ||
| Value | dimSizesBuffer, | ||
| SmallVectorImpl< Value > & | lvlSizesValues, | ||
| Value & | dim2lvlBuffer, | ||
| Value & | lvl2dimBuffer | ||
| ) |
Generates code to set up the buffer parameters for a map.
Definition at line 662 of file CodegenUtils.cpp.
References mlir::Add, allocaBuffer(), constantIndex(), mlir::OpBuilder::create(), mlir::DimId, encodeDim(), encodeLvl(), mlir::presburger::floor(), mlir::FloorDiv, mlir::sparse_tensor::SparseTensorType::getDimRank(), mlir::sparse_tensor::SparseTensorType::getDimToLvl(), mlir::AffineExpr::getKind(), mlir::sparse_tensor::SparseTensorType::getLvlRank(), mlir::sparse_tensor::SparseTensorType::getLvlToDim(), mlir::sparse_tensor::SparseTensorType::isIdentity(), mlir::Mod, mlir::mod(), and mlir::Mul.
| Value mlir::sparse_tensor::genReader | ( | OpBuilder & | builder, |
| Location | loc, | ||
| SparseTensorType | stt, | ||
| Value | tensor, | ||
| SmallVectorImpl< Value > & | dimSizesValues, | ||
| Value & | dimSizesBuffer | ||
| ) |
Generates code that opens a reader and sets the dimension sizes.
Definition at line 616 of file CodegenUtils.cpp.
References allocaBuffer(), constantIndex(), constantPrimaryTypeEncoding(), mlir::OpBuilder::create(), createFuncCall(), mlir::get(), mlir::sparse_tensor::SparseTensorType::getDimRank(), mlir::sparse_tensor::SparseTensorType::getDimShape(), mlir::sparse_tensor::SparseTensorType::getElementType(), mlir::Builder::getIndexType(), getOpaquePointerType(), mlir::sparse_tensor::SparseTensorType::hasDynamicDimShape(), and mlir::sparse_tensor::SparseTensorType::isDynamicDim().
| void mlir::sparse_tensor::genReshapeDstShape | ( | OpBuilder & | builder, |
| Location | loc, | ||
| SmallVectorImpl< Value > & | dstShape, | ||
| ArrayRef< Value > | srcShape, | ||
| ArrayRef< Size > | staticDstShape, | ||
| ArrayRef< ReassociationIndices > | reassociation | ||
| ) |
Computes the shape of destination tensor of a reshape operator.
This is only used when operands have dynamic shape. The shape of the destination is stored into dstShape.
Definition at line 219 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), mlir::detail::enumerate(), and product().
| Value mlir::sparse_tensor::genScalarToTensor | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | elem, | ||
| Type | dstTp | ||
| ) |
Add conversion from scalar to given type (possibly a 0-rank tensor).
Definition at line 166 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), mlir::Type::dyn_cast(), and genCast().
| Value mlir::sparse_tensor::genToCoordinates | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor, | ||
| Level | lvl, | ||
| Level | cooStart | ||
| ) |
Infers the result type and generates ToCoordinatesOp.
If the level is within a COO region, the result type is a memref with unknown stride and offset. Otherwise, the result type is a memref without any specified layout.
Definition at line 566 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), get1DMemRefType(), mlir::Builder::getIndexAttr(), and getSparseTensorType().
Referenced by mlir::sparse_tensor::LoopEmitter::initializeLoopEmit().
| Value mlir::sparse_tensor::genToCoordinatesBuffer | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor | ||
| ) |
Infers the result type and generates ToCoordinatesBufferOp.
Definition at line 575 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), get1DMemRefType(), and getSparseTensorType().
| TypedValue< BaseMemRefType > mlir::sparse_tensor::genToMemref | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor | ||
| ) |
Definition at line 550 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), mlir::get(), and mlir::Value::getType().
| Value mlir::sparse_tensor::genToPositions | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | tensor, | ||
| Level | lvl | ||
| ) |
Infers the result type and generates ToPositionsOp.
Definition at line 557 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), get1DMemRefType(), mlir::Builder::getIndexAttr(), and getSparseTensorType().
Referenced by mlir::sparse_tensor::LoopEmitter::initializeLoopEmit().
Infers the result type and generates ToValuesOp.
Definition at line 583 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), get1DMemRefType(), and getRankedTensorType().
Referenced by mlir::sparse_tensor::LoopEmitter::initializeLoopEmit().
|
inline |
|
inline |
Packs the given values as a "tuple" value.
Definition at line 237 of file SparseTensorDescriptor.h.
References mlir::OpBuilder::create().
Referenced by genTuple(), and mlir::SparseTensorTypeToBufferConverter::SparseTensorTypeToBufferConverter().
Generates code to retrieve the values size for the sparse tensor.
Definition at line 591 of file CodegenUtils.cpp.
References getDescriptorFromTensorTuple(), and mlir::sparse_tensor::SparseTensorDescriptorImpl< ValueArrayRef >::getValMemSize().
|
inline |
Generates a 1D MemRefType with a dynamic size.
When withLayout is set, the returned memref has a layout has unknown strides and offsets. Otherwise, a memref with a standard unit stride zero offset layout is returned.
Definition at line 234 of file CodegenUtils.h.
References mlir::get(), and mlir::Type::getContext().
Referenced by genToCoordinates(), genToCoordinatesBuffer(), genToPositions(), and genToValues().
| SmallVector< unsigned > mlir::sparse_tensor::getBlockSize | ( | AffineMap | dimToLvl | ) |
Given the dimToLvl map, returns the block sizes in a vector.
For instance, a 2x3 block will return [2, 3]. Unblocked dimension i will return 0, and i floordiv 1, i mod 1 will return 1. Therefore, the example below will return [0, 1]. map = ( i, j ) -> ( i : dense, j floordiv 1 : compressed, j mod 1 : dense ) Only valid block sparsity will be accepted.
Definition at line 838 of file SparseTensorDialect.cpp.
References mlir::AffineMap::getResults(), isBlockSparsity(), and mlir::Mod.
|
inline |
Definition at line 248 of file SparseTensorDescriptor.h.
References getTuple().
Referenced by genValMemSize().
|
static |
Definition at line 28 of file Merger.cpp.
References mlir::sparse_tensor::TensorExp::kAbsC, mlir::sparse_tensor::TensorExp::kAbsF, mlir::sparse_tensor::TensorExp::kAbsI, mlir::sparse_tensor::TensorExp::kAddC, mlir::sparse_tensor::TensorExp::kAddF, mlir::sparse_tensor::TensorExp::kAddI, mlir::sparse_tensor::TensorExp::kAndI, mlir::sparse_tensor::TensorExp::kBinary, kBinary, mlir::sparse_tensor::TensorExp::kBinaryBranch, mlir::sparse_tensor::TensorExp::kBitCast, mlir::sparse_tensor::TensorExp::kCastFS, mlir::sparse_tensor::TensorExp::kCastFU, mlir::sparse_tensor::TensorExp::kCastIdx, mlir::sparse_tensor::TensorExp::kCastS, mlir::sparse_tensor::TensorExp::kCastSF, mlir::sparse_tensor::TensorExp::kCastU, mlir::sparse_tensor::TensorExp::kCastUF, mlir::sparse_tensor::TensorExp::kCeilF, mlir::sparse_tensor::TensorExp::kCIm, mlir::sparse_tensor::TensorExp::kCmpF, mlir::sparse_tensor::TensorExp::kCmpI, mlir::sparse_tensor::TensorExp::kCRe, mlir::sparse_tensor::TensorExp::kDenseOp, mlir::sparse_tensor::TensorExp::kDivC, mlir::sparse_tensor::TensorExp::kDivF, mlir::sparse_tensor::TensorExp::kDivS, mlir::sparse_tensor::TensorExp::kDivU, mlir::sparse_tensor::TensorExp::kExpm1C, mlir::sparse_tensor::TensorExp::kExpm1F, mlir::sparse_tensor::TensorExp::kExtF, mlir::sparse_tensor::TensorExp::kFloorF, mlir::sparse_tensor::TensorExp::kInvariant, mlir::sparse_tensor::TensorExp::kLog1pC, mlir::sparse_tensor::TensorExp::kLog1pF, mlir::sparse_tensor::TensorExp::kLoopVar, mlir::sparse_tensor::TensorExp::kMulC, mlir::sparse_tensor::TensorExp::kMulF, mlir::sparse_tensor::TensorExp::kMulI, mlir::sparse_tensor::TensorExp::kNegC, mlir::sparse_tensor::TensorExp::kNegF, mlir::sparse_tensor::TensorExp::kNegI, kNullary, mlir::sparse_tensor::TensorExp::kOrI, mlir::sparse_tensor::TensorExp::kReduce, mlir::sparse_tensor::TensorExp::kSelect, mlir::sparse_tensor::TensorExp::kShlI, mlir::sparse_tensor::TensorExp::kShrS, mlir::sparse_tensor::TensorExp::kShrU, mlir::sparse_tensor::TensorExp::kSinC, mlir::sparse_tensor::TensorExp::kSinF, mlir::sparse_tensor::TensorExp::kSqrtC, mlir::sparse_tensor::TensorExp::kSqrtF, mlir::sparse_tensor::TensorExp::kSubC, mlir::sparse_tensor::TensorExp::kSubF, mlir::sparse_tensor::TensorExp::kSubI, mlir::sparse_tensor::TensorExp::kSynZero, mlir::sparse_tensor::TensorExp::kTanhC, mlir::sparse_tensor::TensorExp::kTanhF, mlir::sparse_tensor::TensorExp::kTensor, mlir::sparse_tensor::TensorExp::kTruncF, mlir::sparse_tensor::TensorExp::kTruncI, mlir::sparse_tensor::TensorExp::kUnary, kUnary, and mlir::sparse_tensor::TensorExp::kXorI.
Referenced by mlir::sparse_tensor::Merger::expContainsTensor(), and mlir::sparse_tensor::Merger::hasNegateOnOut().
| FlatSymbolRefAttr mlir::sparse_tensor::getFunc | ( | ModuleOp | module, |
| StringRef | name, | ||
| TypeRange | resultType, | ||
| ValueRange | operands, | ||
| EmitCInterface | emitCInterface | ||
| ) |
Returns a function reference (first hit also inserts into module).
Sets the "_emit_c_interface" on the function declaration when requested, so that LLVM lowering generates a wrapper function that takes care of ABI complications with passing in and returning MemRefs to C functions.
Definition at line 323 of file CodegenUtils.cpp.
References mlir::OpBuilder::create(), mlir::get(), mlir::ValueRange::getTypes(), and mlir::Operation::setAttr().
Referenced by createFuncCall().
|
constexpr |
Convert a LevelType to its corresponding LevelFormat.
Returns std::nullopt when input lt is Undef.
Definition at line 305 of file Enums.h.
References Undef.
Referenced by getNormalizedEncodingForSpecifier().
|
inline |
Convenience method to abbreviate casting getType().
Definition at line 82 of file SparseTensor.h.
Referenced by getMangledSortHelperFuncName(), and isMatchingWidth().
|
inline |
Definition at line 255 of file SparseTensorDescriptor.h.
References getTuple().
|
inline |
Definition at line 161 of file SparseTensorStorageLayout.h.
References mlir::sparse_tensor::StorageLayout::getNumDataFields().
|
inline |
Definition at line 157 of file SparseTensorStorageLayout.h.
References mlir::sparse_tensor::StorageLayout::getNumFields().
Generates a 1-valued attribute of the given type.
This supports all the same types as getZeroAttr; however, unlike getZeroAttr, for unsupported types we raise llvm_unreachable rather than returning a null attribute.
Definition at line 189 of file CodegenUtils.cpp.
References mlir::DenseElementsAttr::get(), mlir::Builder::getFloatAttr(), mlir::Builder::getIndexAttr(), and mlir::Builder::getIntegerAttr().
Referenced by constantOne().
Definition at line 356 of file CodegenUtils.cpp.
References mlir::Builder::getContext(), and getOpaquePointerType().
| Type mlir::sparse_tensor::getOpaquePointerType | ( | MLIRContext * | ctx | ) |
Returns the equivalent of void* for opaque arguments to the execution engine.
Definition at line 352 of file CodegenUtils.cpp.
References mlir::get().
Referenced by genReader(), and getOpaquePointerType().
| Type mlir::sparse_tensor::getOverheadType | ( | Builder & | builder, |
| OverheadType | ot | ||
| ) |
Converts the internal type-encoding for overhead storage to an mlir::Type.
Definition at line 54 of file CodegenUtils.cpp.
References mlir::Builder::getIndexType(), mlir::Builder::getIntegerType(), kIndex, kU16, kU32, kU64, and kU8.
|
inline |
Convenience method to abbreviate casting getType().
Definition at line 74 of file SparseTensor.h.
Referenced by foreachInSparseConstant(), genToValues(), and mlir::sparse_tensor::LoopEmitter::initialize().
| SparseTensorEncodingAttr mlir::sparse_tensor::getSparseTensorEncoding | ( | Type | type | ) |
Convenience method to get a sparse encoding attribute from a type.
Returns null-attribute for any type without an encoding.
Definition at line 762 of file SparseTensorDialect.cpp.
Referenced by containsSparseTensor(), createOrFoldSliceOffsetOp(), createOrFoldSliceStrideOp(), findSparseAnnotations(), flattenOperands(), genSliceOffset(), genSliceStride(), isSparseTensor(), mlir::sparse_tensor::IterationGraphSorter::sort(), mlir::SparseTensorTypeToBufferConverter::SparseTensorTypeToBufferConverter(), and mlir::sparse_tensor::CodegenEnv::startEmit().
|
inline |
Convenience methods to obtain a SparseTensorType from a Value.
Definition at line 344 of file SparseTensorType.h.
References mlir::Value::getType().
Referenced by genIf(), genIndex(), genSubscript(), genToCoordinates(), genToCoordinatesBuffer(), genToPositions(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), mlir::sparse_tensor::CodegenEnv::isAdmissibleTensorExp(), lvlIsInBounds(), and sizesFromSrc().
Scans to top of generated loop.
Definition at line 422 of file CodegenUtils.cpp.
|
inline |
Returns the "tuple" value of the adapted tensor.
Definition at line 232 of file SparseTensorDescriptor.h.
References mlir::Value::getDefiningOp().
Referenced by flattenOperands(), getDescriptorFromTensorTuple(), and getMutDescriptorFromTensorTuple().
| bool mlir::sparse_tensor::hasAnyNonIdentityOperandsOrResults | ( | Operation * | op | ) |
Returns true iff MLIR operation has any sparse tensor with non-identity dim2lvl maps.
Definition at line 885 of file SparseTensorDialect.cpp.
Referenced by mlir::sparse_tensor::IterationGraphSorter::fromGenericOp().
|
inline |
Returns true iff MLIR operand has any sparse operand.
Definition at line 93 of file SparseTensor.h.
|
inline |
Returns true iff MLIR operand has any sparse operand or result.
Definition at line 107 of file SparseTensor.h.
Referenced by mlir::sparse_tensor::IterationGraphSorter::fromGenericOp().
|
inline |
Returns true iff MLIR operand has any sparse result.
Definition at line 100 of file SparseTensor.h.
| AffineMap mlir::sparse_tensor::inferLvlToDim | ( | AffineMap | dimToLvl, |
| MLIRContext * | context | ||
| ) |
Given the dimToLvl map, infers the lvlToDim map, or returns empty Affine map when inference fails.
Definition at line 770 of file SparseTensorDialect.cpp.
References mlir::AffineMap::getNumSymbols(), inverseBlockSparsity(), mlir::inversePermutation(), isBlockSparsity(), and mlir::AffineMap::isPermutation().
Referenced by genReplaceDimToLvlMap().
|
static |
Definition at line 1493 of file Merger.cpp.
References mlir::Region::begin(), mlir::Region::cloneInto(), mlir::OpBuilder::create(), mlir::RewriterBase::eraseOp(), mlir::Region::front(), mlir::Block::getTerminator(), and mlir::RewriterBase::inlineBlockBefore().
Referenced by mlir::sparse_tensor::Merger::buildExp().
| AffineMap mlir::sparse_tensor::inverseBlockSparsity | ( | AffineMap | dimToLvl, |
| MLIRContext * | context | ||
| ) |
Returns the lvlToDim map for the given dimToLvl map specific to the block sparse cases.
Asserts on failure (so only use when known to succeed).
Definition at line 785 of file SparseTensorDialect.cpp.
References mlir::Add, mlir::FloorDiv, mlir::AffineMap::get(), mlir::getAffineBinaryOpExpr(), mlir::getAffineDimExpr(), mlir::AffineMap::getNumResults(), mlir::AffineMap::getResult(), mlir::Mod, and mlir::Mul.
Referenced by inferLvlToDim().
|
constexpr |
Check if the LevelType is 2OutOf4 (regardless of properties).
Definition at line 277 of file Enums.h.
References TwoOutOfFour.
Referenced by allocSchemeForRank(), genEndInsert(), genIf(), mlir::sparse_tensor::Merger::hasAnySparse(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), mlir::sparse_tensor::SparseTensorType::is2OutOf4Lvl(), mlir::sparse_tensor::SparseTensorStorageBase::is2OutOf4Lvl(), mlir::sparse_tensor::Merger::isSparseLvlWithNonTrivialIdxExp(), isWithCrdLT(), mlir::sparse_tensor::Merger::simplifyCond(), and startLoopSeq().
Ensures that the sparsifier can generate code for branch.
Definition at line 1242 of file Merger.cpp.
|
static |
Ensures that the sparsifier can generate code for expression.
Definition at line 1222 of file Merger.cpp.
| bool mlir::sparse_tensor::isBlockSparsity | ( | AffineMap | dimToLvl | ) |
Given the dimToLvl map, returns if it's block sparsity.
Definition at line 855 of file SparseTensorDialect.cpp.
References mlir::FloorDiv, mlir::AffineMap::getResults(), and mlir::Mod.
Referenced by getBlockSize(), and inferLvlToDim().
|
constexpr |
Definition at line 140 of file Enums.h.
Referenced by mlir::sparse_tensor::SparseTensorReader::canReadAs().
|
constexpr |
Check if the LevelType is compressed (regardless of properties).
Definition at line 259 of file Enums.h.
References Compressed.
Referenced by allocSchemeForRank(), createAllocFields(), genEndInsert(), genIf(), mlir::sparse_tensor::Merger::hasAnySparse(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), mlir::sparse_tensor::SparseTensorType::isCompressedLvl(), mlir::sparse_tensor::SparseTensorStorageBase::isCompressedLvl(), mlir::sparse_tensor::Merger::isSparseLvlWithNonTrivialIdxExp(), isWithCrdLT(), isWithPosLT(), mlir::sparse_tensor::Merger::simplifyCond(), and startLoopSeq().
|
constexpr |
Check if the LevelType is dense (regardless of properties).
Definition at line 253 of file Enums.h.
References Dense.
Referenced by allocSchemeForRank(), mlir::sparse_tensor::LoopEmitter::enterFilterLoopOverTensorAtLvl(), findAffine(), mlir::sparse_tensor::LoopEmitter::genDenseAffineAddress(), genEndInsert(), genIf(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), isAllDense(), mlir::sparse_tensor::SparseTensorType::isDenseLvl(), mlir::sparse_tensor::SparseTensorStorageBase::isDenseLvl(), mlir::sparse_tensor::Merger::simplifyCond(), startLoopSeq(), and translateBitsToTidLvlPairs().
|
constexpr |
|
constexpr |
|
constexpr |
|
constexpr |
Definition at line 128 of file Enums.h.
Referenced by mlir::sparse_tensor::SparseTensorReader::canReadAs().
|
constexpr |
|
constexpr |
Check if the LevelType is loose compressed (regardless of properties).
Definition at line 271 of file Enums.h.
References LooseCompressed.
Referenced by allocSchemeForRank(), createAllocFields(), genEndInsert(), genIf(), mlir::sparse_tensor::Merger::hasAnySparse(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), mlir::sparse_tensor::SparseTensorType::isLooseCompressedLvl(), mlir::sparse_tensor::SparseTensorStorageBase::isLooseCompressedLvl(), mlir::sparse_tensor::Merger::isSparseLvlWithNonTrivialIdxExp(), isWithCrdLT(), isWithPosLT(), mlir::sparse_tensor::Merger::simplifyCond(), and startLoopSeq().
|
constexpr |
Check if the LevelType is ordered (regardless of storage format).
Definition at line 294 of file Enums.h.
Referenced by mlir::sparse_tensor::SparseTensorType::isOrderedLvl(), and mlir::sparse_tensor::SparseTensorStorageBase::isOrderedLvl().
|
constexpr |
Definition at line 136 of file Enums.h.
Referenced by mlir::sparse_tensor::SparseTensorReader::canReadAs().
|
constexpr |
Check if the LevelType is singleton (regardless of properties).
Definition at line 265 of file Enums.h.
References Singleton.
Referenced by allocSchemeForRank(), genEndInsert(), genIf(), mlir::sparse_tensor::Merger::hasAnySparse(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), mlir::sparse_tensor::SparseTensorType::isSingletonLvl(), mlir::sparse_tensor::SparseTensorStorageBase::isSingletonLvl(), mlir::sparse_tensor::Merger::isSparseLvlWithNonTrivialIdxExp(), isWithCrdLT(), mlir::sparse_tensor::Merger::simplifyCond(), and startLoopSeq().
|
constexpr |
Check if the LevelType is the special undefined value.
Definition at line 250 of file Enums.h.
References Undef.
Referenced by findAffine(), findDepIdxSet(), genIf(), mlir::sparse_tensor::ir_detail::LvlSpec::LvlSpec(), startLoopSeq(), and translateBitsToTidLvlPairs().
|
constexpr |
Check if the LevelType is unique (regardless of storage format).
Definition at line 299 of file Enums.h.
Referenced by mlir::sparse_tensor::SparseTensorType::isUniqueLvl(), mlir::sparse_tensor::SparseTensorStorageBase::isUniqueLvl(), and translateBitsToTidLvlPairs().
|
constexpr |
Check that the LevelType contains a valid (possibly undefined) value.
Definition at line 239 of file Enums.h.
Referenced by buildLevelType(), mlir::sparse_tensor::ir_detail::LvlSpec::LvlSpec(), mlir::sparse_tensor::ir_detail::LvlTypeParser::parseLvlType(), and mlir::sparse_tensor::Merger::setLevelAndType().
|
constexpr |
Check if the LevelType needs coordinates array.
Definition at line 288 of file Enums.h.
References is2OutOf4LT(), isCompressedLT(), isLooseCompressedLT(), and isSingletonLT().
Referenced by mlir::sparse_tensor::StorageLayout::foreachField(), and mlir::sparse_tensor::SparseTensorType::isWithCrd().
|
constexpr |
Check if the LevelType needs positions array.
Definition at line 283 of file Enums.h.
References isCompressedLT(), and isLooseCompressedLT().
Referenced by mlir::sparse_tensor::StorageLayout::foreachField(), and mlir::sparse_tensor::SparseTensorType::isWithPos().
|
inline |
Definition at line 429 of file CodegenUtils.h.
Referenced by mlir::sparse_tensor::LoopEmitter::initialize().
|
static |
Definition at line 692 of file Merger.cpp.
Referenced by mlir::sparse_tensor::Merger::dumpExp().
| SmallVector< Value > mlir::sparse_tensor::loadAll | ( | OpBuilder & | builder, |
| Location | loc, | ||
| size_t | size, | ||
| Value | mem, | ||
| size_t | offsetIdx = 0, |
||
| Value | offsetVal = Value() |
||
| ) |
Loads size-many values from the memref, which must have rank-1 and size greater-or-equal to size.
If the optional (offsetIdx,offsetVal) arguments are provided, then the offsetVal will be added to the offsetIdx-th value after loading.
Definition at line 507 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), and mlir::Value::getType().
| OverheadType mlir::sparse_tensor::overheadTypeEncoding | ( | Type | tp | ) |
Converts an overhead storage type to its internal type-encoding.
Definition at line 46 of file CodegenUtils.cpp.
References mlir::Type::isIndex(), kIndex, and overheadTypeEncoding().
| OverheadType mlir::sparse_tensor::overheadTypeEncoding | ( | unsigned | width | ) |
Converts an overhead storage bitwidth to its internal type-encoding.
Definition at line 30 of file CodegenUtils.cpp.
References kIndex, kU16, kU32, kU64, and kU8.
Referenced by constantOverheadTypeEncoding(), crdTypeEncoding(), overheadTypeEncoding(), overheadTypeFunctionSuffix(), and posTypeEncoding().
| StringRef mlir::sparse_tensor::overheadTypeFunctionSuffix | ( | OverheadType | ot | ) |
Convert OverheadType to its function-name suffix.
Definition at line 88 of file CodegenUtils.cpp.
References CASE, kIndex, and MLIR_SPARSETENSOR_FOREVERY_FIXED_O.
Referenced by overheadTypeFunctionSuffix().
| StringRef mlir::sparse_tensor::overheadTypeFunctionSuffix | ( | Type | overheadTp | ) |
Converts an overhead storage type to its function-name suffix.
Definition at line 101 of file CodegenUtils.cpp.
References overheadTypeEncoding(), and overheadTypeFunctionSuffix().
| OverheadType mlir::sparse_tensor::posTypeEncoding | ( | SparseTensorEncodingAttr | enc | ) |
Returns the OverheadType for position overhead storage.
Definition at line 71 of file CodegenUtils.cpp.
References overheadTypeEncoding().
| PrimaryType mlir::sparse_tensor::primaryTypeEncoding | ( | Type | elemTp | ) |
Converts a primary storage type to its internal type-encoding.
Definition at line 105 of file CodegenUtils.cpp.
References mlir::Type::isBF16(), mlir::Type::isF16(), mlir::Type::isF32(), mlir::Type::isF64(), mlir::Type::isInteger(), kBF16, kC32, kC64, kF16, kF32, kF64, kI16, kI32, kI64, and kI8.
Referenced by constantPrimaryTypeEncoding(), and primaryTypeFunctionSuffix().
| StringRef mlir::sparse_tensor::primaryTypeFunctionSuffix | ( | PrimaryType | pt | ) |
Convert PrimaryType to its function-name suffix.
Definition at line 132 of file CodegenUtils.cpp.
References CASE, and MLIR_SPARSETENSOR_FOREVERY_V.
Referenced by primaryTypeFunctionSuffix().
| StringRef mlir::sparse_tensor::primaryTypeFunctionSuffix | ( | Type | elemTp | ) |
Converts a primary storage type to its function-name suffix.
Definition at line 143 of file CodegenUtils.cpp.
References primaryTypeEncoding(), and primaryTypeFunctionSuffix().
| void mlir::sparse_tensor::registerBufferizableOpInterfaceExternalModels | ( | DialectRegistry & | registry | ) |
Definition at line 319 of file BufferizableOpInterfaceImpl.cpp.
References mlir::DialectRegistry::addExtension().
Referenced by mlir::registerAllDialects().
| void mlir::sparse_tensor::registerSparseTensorPipelines | ( | ) |
Registers all pipelines for the sparse_tensor dialect.
At present, this includes only "sparsifier".
Definition at line 111 of file SparseTensorPipelines.cpp.
References buildSparsifier().
Referenced by mlir::registerAllPasses().
| void mlir::sparse_tensor::registerTransformDialectExtension | ( | DialectRegistry & | registry | ) |
Definition at line 54 of file SparseTensorTransformOps.cpp.
References mlir::DialectRegistry::addExtensions().
Referenced by mlir::registerAllExtensions().
| void mlir::sparse_tensor::reshapeCvs | ( | OpBuilder & | builder, |
| Location | loc, | ||
| ArrayRef< ReassociationIndices > | reassociation, | ||
| ValueRange | srcSizes, | ||
| ValueRange | srcCvs, | ||
| ValueRange | dstSizes, | ||
| SmallVectorImpl< Value > & | dstCvs | ||
| ) |
Reshape coordinates during a reshaping operation.
Definition at line 275 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), and mlir::detail::enumerate().
| void mlir::sparse_tensor::sizesFromSrc | ( | OpBuilder & | builder, |
| SmallVectorImpl< Value > & | sizes, | ||
| Location | loc, | ||
| Value | src | ||
| ) |
Populates given sizes array from dense tensor or sparse tensor constant.
Definition at line 414 of file CodegenUtils.cpp.
References mlir::linalg::createOrFoldDimOp(), mlir::sparse_tensor::SparseTensorType::getDimRank(), and getSparseTensorType().
Referenced by concatSizesFromInputs().
| void mlir::sparse_tensor::storeAll | ( | OpBuilder & | builder, |
| Location | loc, | ||
| Value | mem, | ||
| ValueRange | vs, | ||
| size_t | offsetIdx = 0, |
||
| Value | offsetVal = Value() |
||
| ) |
Stores all the values of vs into the memref mem, which must have rank-1 and size greater-or-equal to vs.size().
If the optional (offsetIdx,offsetVal) arguments are provided, then the offsetVal will be added to the offsetIdx-th value before storing.
Definition at line 529 of file CodegenUtils.cpp.
References constantIndex(), mlir::OpBuilder::create(), mlir::detail::enumerate(), and mlir::Value::getType().
Convenience method to translate the given level to the corresponding dimension.
Requires: enc has a permuted dim2lvl map and 0 <= l < lvlRank.
Definition at line 895 of file SparseTensorDialect.cpp.
References mlir::AffineMap::getDimPosition().
Referenced by genSliceOffset(), and genSliceStride().
|
inline |
Definition at line 99 of file SparseTensorStorageLayout.h.
Convenience method to translate the given dimension to the corresponding level.
Requires: enc has a permuted dim2lvl map and 0 <= d < dimRank.
Definition at line 904 of file SparseTensorDialect.cpp.
References mlir::AffineMap::getDimPosition().
|
constexpr |
Returns string representation of the given dimension level type.
Definition at line 202 of file Enums.h.
References Compressed, CompressedNo, CompressedNu, CompressedNuNo, Dense, LooseCompressed, LooseCompressedNo, LooseCompressedNu, LooseCompressedNuNo, Singleton, SingletonNo, SingletonNu, SingletonNuNo, TwoOutOfFour, and Undef.
Referenced by mlir::sparse_tensor::Merger::dumpBits().
|
inline |
Definition at line 94 of file SparseTensorStorageLayout.h.
|
inline |
Definition at line 347 of file SparseTensorType.h.
References mlir::Value::getType().
1.9.1