Dialect Conversion

This document describes a framework in MLIR in which to perform operation conversions between, and within dialects. This framework allows for transforming illegal operations to those supported by a provided conversion target, via a set of pattern-based operation rewriting patterns.

The dialect conversion framework consists of the following components:

• A Conversion Target
• A set of Rewrite Patterns
• A Type Converter (Optional)

Modes of Conversion ¶

When applying a conversion to a set of operations, there are several different conversion modes that may be selected from:

• Partial Conversion

  • A partial conversion will legalize as many operations to the target as possible, but will allow pre-existing operations that were not explicitly marked as “illegal” to remain unconverted. This allows for partially lowering parts of the input in the presence of unknown operations.
  • A partial conversion can be applied via applyPartialConversion.

• Full Conversion

  • A full conversion legalizes all input operations, and is only successful if all operations are properly legalized to the given conversion target. This ensures that only known operations will exist after the conversion process.
  • A full conversion can be applied via applyFullConversion.

• Analysis Conversion

  • An analysis conversion will analyze which operations are legalizable to the given conversion target if a conversion were to be applied. This is done by performing a ‘partial’ conversion and recording which operations would have been successfully converted if successful. Note that no rewrites, or transformations, are actually applied to the input operations.
  • An analysis conversion can be applied via applyAnalysisConversion.

In all cases, the framework walks the operations in preorder, examining an op before the ops in any regions it has.

Conversion Target ¶

The conversion target is a formal definition of what is considered to be legal during the conversion process. The final operations generated by the conversion framework must be marked as legal on the ConversionTarget for the rewrite to be a success. Depending on the conversion mode, existing operations need not always be legal. Operations and dialects may be marked with any of the provided legality actions below:

• Legal

  • This action signals that every instance of a given operation is legal, i.e. any combination of attributes, operands, types, etc. are valid.

• Dynamic

  • This action signals that only some instances of a given operation are legal. This allows for defining fine-tune constraints, e.g. saying that arith.addi is only legal when operating on 32-bit integers.

• Illegal

  • This action signals that no instance of a given operation is legal. Operations marked as “illegal” must always be converted for the conversion to be successful. This action also allows for selectively marking specific operations as illegal in an otherwise legal dialect.

Operations and dialects that are neither explicitly marked legal nor illegal are separate from the above (“unknown” operations) and are treated differently, for example, for the purposes of partial conversion as mentioned above.

An example conversion target is shown below:

struct MyTarget : public ConversionTarget {
  MyTarget(MLIRContext &ctx) : ConversionTarget(ctx) {
    //--------------------------------------------------------------------------
    // Marking an operation as Legal:

    /// Mark all operations within the LLVM dialect are legal.
    addLegalDialect<LLVMDialect>();

    /// Mark `arith.constant` op is always legal on this target.
    addLegalOp<arith::ConstantOp>();

    //--------------------------------------------------------------------------
    // Marking an operation as dynamically legal.

    /// Mark all operations within Affine dialect have dynamic legality
    /// constraints.
    addDynamicallyLegalDialect<affine::AffineDialect>(
        [](Operation *op) { ... });

    /// Mark `func.return` as dynamically legal, but provide a specific legality
    /// callback.
    addDynamicallyLegalOp<func::ReturnOp>([](func::ReturnOp op) { ... });

    /// Treat unknown operations, i.e. those without a legalization action
    /// directly set, as dynamically legal.
    markUnknownOpDynamicallyLegal([](Operation *op) { ... });

    //--------------------------------------------------------------------------
    // Marking an operation as illegal.

    /// All operations within the GPU dialect are illegal.
    addIllegalDialect<GPUDialect>();

    /// Mark `cf.br` and `cf.cond_br` as illegal.
    addIllegalOp<cf::BranchOp, cf::CondBranchOp>();
  }

  /// Implement the default legalization handler to handle operations marked as
  /// dynamically legal that were not provided with an explicit handler.
  bool isDynamicallyLegal(Operation *op) override { ... }
};

Recursive Legality ¶

In some cases, it may be desirable to mark entire regions as legal. This provides an additional granularity of context to the concept of “legal”. If an operation is marked recursively legal, either statically or dynamically, then all of the operations nested within are also considered legal even if they would otherwise be considered “illegal”. An operation can be marked via markOpRecursivelyLegal<>:

ConversionTarget &target = ...;

/// The operation must first be marked as `Legal` or `Dynamic`.
target.addLegalOp<MyOp>(...);
target.addDynamicallyLegalOp<MySecondOp>(...);

/// Mark the operation as always recursively legal.
target.markOpRecursivelyLegal<MyOp>();
/// Mark optionally with a callback to allow selective marking.
target.markOpRecursivelyLegal<MyOp, MySecondOp>([](Operation *op) { ... });
/// Mark optionally with a callback to allow selective marking.
target.markOpRecursivelyLegal<MyOp>([](MyOp op) { ... });

Rewrite Pattern Specification ¶

After the conversion target has been defined, a set of legalization patterns must be provided to transform illegal operations into legal ones. The patterns supplied here have the same structure and restrictions as those described in the main Pattern documentation. The patterns provided do not need to generate operations that are directly legal on the target. The framework will automatically build a graph of conversions to convert non-legal operations into a set of legal ones.

As an example, say you define a target that supports one operation: foo.add. When providing the following patterns: [bar.add -> baz.add, baz.add -> foo.add], the framework will automatically detect that it can legalize bar.add -> foo.add even though a direct conversion does not exist. This means that you don’t have to define a direct legalization pattern for bar.add -> foo.add.

Conversion Patterns ¶

Along with the general RewritePattern classes, the conversion framework provides a special type of rewrite pattern that can be used when a pattern relies on interacting with constructs specific to the conversion process, the ConversionPattern. For example, the conversion process does not necessarily update operations in-place and instead creates a mapping of events such as replacements and erasures, and only applies them when the entire conversion process is successful. Certain classes of patterns rely on using the updated/remapped operands of an operation, such as when the types of results defined by an operation have changed. The general Rewrite Patterns can no longer be used in these situations, as the types of the operands of the operation being matched will not correspond with those expected by the user. This pattern provides, as an additional argument to the matchAndRewrite and rewrite methods, the list of operands that the operation should use after conversion. If an operand was the result of a non-converted operation, for example if it was already legal, the original operand is used. This means that the operands provided always have a 1-1 non-null correspondence with the operands on the operation. The original operands of the operation are still intact and may be inspected as normal. These patterns also utilize a special PatternRewriter, ConversionPatternRewriter, that provides special hooks for use with the conversion infrastructure.

struct MyConversionPattern : public ConversionPattern {
  /// The `matchAndRewrite` hooks on ConversionPatterns take an additional
  /// `operands` parameter, containing the remapped operands of the original
  /// operation.
  virtual LogicalResult
  matchAndRewrite(Operation *op, ArrayRef<Value> operands,
                  ConversionPatternRewriter &rewriter) const;
};

Type Safety ¶

The types of the remapped operands provided to a conversion pattern must be of a type expected by the pattern. The expected types of a pattern are determined by a provided TypeConverter. If no type converter is provided, the types of the remapped operands are expected to match the types of the original operands. If a type converter is provided, the types of the remapped operands are expected to be legal as determined by the converter. If the remapped operand types are not of an expected type, and a materialization to the expected type could not be performed, the pattern fails application before the matchAndRewrite hook is invoked. This ensures that patterns do not have to explicitly ensure type safety, or sanitize the types of the incoming remapped operands. More information on type conversion is detailed in the dedicated section below.

Type Conversion ¶

It is sometimes necessary as part of a conversion to convert the set types of being operated on. In these cases, a TypeConverter object may be defined that details how types should be converted when interfacing with a pattern. A TypeConverter may be used to convert the signatures of block arguments and regions, to define the expected inputs types of the pattern, and to reconcile type differences in general.

Type Converter ¶

The TypeConverter contains several hooks for detailing how to convert types, and how to materialize conversions between types in various situations. The two main aspects of the TypeConverter are conversion and materialization.

A conversion describes how a given source Type should be converted to N target types. If the source type is converted to itself, we say it is a “legal” type. Type conversions are specified via the addConversion method described below.

A materialization describes how a list of values should be converted to a list of values with specific types. An important distinction from a conversion is that a materialization can produce IR, whereas a conversion cannot. These materializations are used by the conversion framework to ensure type safety during the conversion process. There are several types of materializations depending on the situation.

• Argument Materialization

  • An argument materialization is used when converting the type of a block argument during a signature conversion. The new block argument types are specified in a SignatureConversion object. An original block argument can be converted into multiple block arguments, which is not supported everywhere in the dialect conversion. (E.g., adaptors support only a single replacement value for each original value.) Therefore, an argument materialization is used to convert potentially multiple new block arguments back into a single SSA value. An argument materialization is also used when replacing an op result with multiple values.

• Source Materialization

  • A source materialization is used when a value was replaced with a value of a different type, but there are still users that expects the original (“source”) type at the end of the conversion process. A source materialization converts the replacement value back to the source type.
  • This materialization is used in the following situations:
    • When a block argument has been converted to a different type, but the original argument still has users that will remain live after the conversion process has finished.
    • When a block argument has been dropped, but the argument still has users that will remain live after the conversion process has finished.
    • When the result type of an operation has been converted to a different type, but the original result still has users that will remain live after the conversion process is finished.

• Target Materialization

  • A target materialization converts a value to the type that is expected by a conversion pattern according to its type converter.
  • A target materialization is used when a pattern expects the remapped operands to be of a certain set of types, but the original input operands have either not been replaced or been replaced with values of a different type.

If a converted value is used by an operation that isn’t converted, it needs a conversion back to the source type, hence source materialization; if an unconverted value is used by an operation that is being converted, it needs conversion to the target type, hence target materialization.

As noted above, the conversion process guarantees that the type contract of the IR is preserved during the conversion. This means that the types of value uses will not implicitly change during the conversion process. When the type of a value definition, either block argument or operation result, is being changed, the users of that definition must also be updated during the conversion process. If they aren’t, a type conversion must be materialized to ensure that a value of the expected type is still present within the IR. If a materialization is required, but cannot be performed, the entire conversion process fails.

Several of the available hooks are detailed below:

class TypeConverter {
 public:
  /// Register a conversion function. A conversion function defines how a given
  /// source type should be converted. A conversion function must be convertible
  /// to any of the following forms(where `T` is a class derived from `Type`:
  ///   * Optional<Type>(T)
  ///     - This form represents a 1-1 type conversion. It should return nullptr
  ///       or `std::nullopt` to signify failure. If `std::nullopt` is returned, the
  ///       converter is allowed to try another conversion function to perform
  ///       the conversion.
  ///   * Optional<LogicalResult>(T, SmallVectorImpl<Type> &)
  ///     - This form represents a 1-N type conversion. It should return
  ///       `failure` or `std::nullopt` to signify a failed conversion. If the new
  ///       set of types is empty, the type is removed and any usages of the
  ///       existing value are expected to be removed during conversion. If
  ///       `std::nullopt` is returned, the converter is allowed to try another
  ///       conversion function to perform the conversion.
  ///   * Optional<LogicalResult>(T, SmallVectorImpl<Type> &, ArrayRef<Type>)
  ///     - This form represents a 1-N type conversion supporting recursive
  ///       types. The first two arguments and the return value are the same as
  ///       for the regular 1-N form. The third argument is contains is the
  ///       "call stack" of the recursive conversion: it contains the list of
  ///       types currently being converted, with the current type being the
  ///       last one. If it is present more than once in the list, the
  ///       conversion concerns a recursive type.
  /// Note: When attempting to convert a type, e.g. via 'convertType', the
  ///       mostly recently added conversions will be invoked first.
  template <typename FnT,
            typename T = typename llvm::function_traits<FnT>::template arg_t<0>>
  void addConversion(FnT &&callback) {
    registerConversion(wrapCallback<T>(std::forward<FnT>(callback)));
  }

  /// All of the following materializations require function objects that are
  /// convertible to the following form:
  ///   `std::optional<Value>(OpBuilder &, T, ValueRange, Location)`,
  /// where `T` is any subclass of `Type`. This function is responsible for
  /// creating an operation, using the OpBuilder and Location provided, that
  /// "casts" a range of values into a single value of the given type `T`. It
  /// must return a Value of the converted type on success, an `std::nullopt` if
  /// it failed but other materialization can be attempted, and `nullptr` on
  /// unrecoverable failure. It will only be called for (sub)types of `T`.
  /// Materialization functions must be provided when a type conversion may
  /// persist after the conversion has finished.

  /// This method registers a materialization that will be called when
  /// converting (potentially multiple) block arguments that were the result of
  /// a signature conversion of a single block argument, to a single SSA value
  /// with the old argument type.
  template <typename FnT,
            typename T = typename llvm::function_traits<FnT>::template arg_t<1>>
  void addArgumentMaterialization(FnT &&callback) {
    argumentMaterializations.emplace_back(
        wrapMaterialization<T>(std::forward<FnT>(callback)));
  }

  /// This method registers a materialization that will be called when
  /// converting a replacement value back to its original source type.
  /// This is used when some uses of the original value persist beyond the main
  /// conversion.
  template <typename FnT,
            typename T = typename llvm::function_traits<FnT>::template arg_t<1>>
  void addSourceMaterialization(FnT &&callback) {
    sourceMaterializations.emplace_back(
        wrapMaterialization<T>(std::forward<FnT>(callback)));
  }

  /// This method registers a materialization that will be called when
  /// converting a value to a target type according to a pattern's type
  /// converter.
  ///
  /// Note: Target materializations can optionally inspect the "original"
  /// type. This type may be different from the type of the input value.
  /// For example, let's assume that a conversion pattern "P1" replaced an SSA
  /// value "v1" (type "t1") with "v2" (type "t2"). Then a different conversion
  /// pattern "P2" matches an op that has "v1" as an operand. Let's furthermore
  /// assume that "P2" determines that the converted target type of "t1" is
  /// "t3", which may be different from "t2". In this example, the target
  /// materialization will be invoked with: outputType = "t3", inputs = "v2",
  /// originalType = "t1". Note that the original type "t1" cannot be recovered
  /// from just "t3" and "v2"; that's why the originalType parameter exists.
  ///
  /// Note: During a 1:N conversion, the result types can be a TypeRange. In
  /// that case the materialization produces a SmallVector<Value>.
  template <typename FnT,
            typename T = typename llvm::function_traits<FnT>::template arg_t<1>>
  void addTargetMaterialization(FnT &&callback) {
    targetMaterializations.emplace_back(
        wrapMaterialization<T>(std::forward<FnT>(callback)));
  }
};

Materializations through the type converter are optional. If the ConversionConfig::buildMaterializations flag is set to “false”, the dialect conversion driver builds an unrealized_conversion_cast op instead of calling the respective type converter callback whenever a materialization is required.

Region Signature Conversion ¶

From the perspective of type conversion, the types of block arguments are a bit special. Throughout the conversion process, blocks may move between regions of different operations. Given this, the conversion of the types for blocks must be done explicitly via a conversion pattern.

To convert the types of block arguments within a Region, a custom hook on the ConversionPatternRewriter must be invoked; convertRegionTypes. This hook uses a provided type converter to apply type conversions to all blocks of a given region. As noted above, the conversions performed by this method use the argument materialization hook on the TypeConverter. This hook also takes an optional TypeConverter::SignatureConversion parameter that applies a custom conversion to the entry block of the region. The types of the entry block arguments are often tied semantically to the operation, e.g., func::FuncOp, AffineForOp, etc.

To convert the signature of just one given block, the applySignatureConversion hook can be used.

A signature conversion, TypeConverter::SignatureConversion, can be built programmatically:

class SignatureConversion {
public:
    /// Remap an input of the original signature with a new set of types. The
    /// new types are appended to the new signature conversion.
    void addInputs(unsigned origInputNo, ArrayRef<Type> types);

    /// Append new input types to the signature conversion, this should only be
    /// used if the new types are not intended to remap an existing input.
    void addInputs(ArrayRef<Type> types);

    /// Remap an input of the original signature with a range of types in the
    /// new signature.
    void remapInput(unsigned origInputNo, unsigned newInputNo,
                    unsigned newInputCount = 1);

    /// Remap an input of the original signature to another `replacement`
    /// value. This drops the original argument.
    void remapInput(unsigned origInputNo, Value replacement);
};

The TypeConverter provides several default utilities for signature conversion and legality checking: convertSignatureArgs/convertBlockSignature/isLegal(Region *|Type).

Debugging ¶

To debug the execution of the dialect conversion framework, -debug-only=dialect-conversion may be used. This command line flag activates LLVM’s debug logging infrastructure solely for the conversion framework. The output is formatted as a tree structure, mirroring the structure of the conversion process. This output contains all of the actions performed by the rewriter, how generated operations get legalized, and why they fail.

Example output is shown below:

//===-------------------------------------------===//
Legalizing operation : 'func.return'(0x608000002e20) {
  "func.return"() : () -> ()

  * Fold {
  } -> FAILURE : unable to fold

  * Pattern : 'func.return -> ()' {
    ** Insert  : 'spirv.Return'(0x6070000453e0)
    ** Replace : 'func.return'(0x608000002e20)

    //===-------------------------------------------===//
    Legalizing operation : 'spirv.Return'(0x6070000453e0) {
      "spirv.Return"() : () -> ()

    } -> SUCCESS : operation marked legal by the target
    //===-------------------------------------------===//
  } -> SUCCESS : pattern applied successfully
} -> SUCCESS
//===-------------------------------------------===//

This output is describing the legalization of an func.return operation. We first try to legalize by folding the operation, but that is unsuccessful for func.return. From there, a pattern is applied that replaces the func.return with a spirv.Return. The newly generated spirv.Return is then processed for legalization, but is found to already legal as per the target.