MLIR  19.0.0git
Public Types | Public Member Functions | Static Public Member Functions | List of all members
mlir::Token Class Reference

This represents a token in the MLIR syntax. More...

#include "AsmParser/Token.h"

Public Types

enum  Kind
 

Public Member Functions

 Token (Kind kind, StringRef spelling)
 
StringRef getSpelling () const
 
Kind getKind () const
 
bool is (Kind k) const
 
bool isAny (Kind k1, Kind k2) const
 
template<typename... T>
bool isAny (Kind k1, Kind k2, Kind k3, T... others) const
 Return true if this token is one of the specified kinds. More...
 
bool isNot (Kind k) const
 
template<typename... T>
bool isNot (Kind k1, Kind k2, T... others) const
 Return true if this token isn't one of the specified kinds. More...
 
bool isKeyword () const
 Return true if this is one of the keyword token kinds (e.g. kw_if). More...
 
bool isCodeCompletion () const
 Returns true if the current token represents a code completion. More...
 
bool isCodeCompletionFor (Kind kind) const
 Returns true if the current token represents a code completion for the "normal" token type. More...
 
bool isOrIsCodeCompletionFor (Kind kind) const
 Returns true if the current token is the given type, or represents a code completion for that type. More...
 
std::optional< unsigned > getUnsignedIntegerValue () const
 For an integer token, return its value as an unsigned. More...
 
std::optional< uint64_t > getUInt64IntegerValue () const
 
std::optional< double > getFloatingPointValue () const
 For a floatliteral token, return its value as a double. More...
 
std::optional< unsigned > getIntTypeBitwidth () const
 For an inttype token, return its bitwidth. More...
 
std::optional< bool > getIntTypeSignedness () const
 For an inttype token, return its signedness semantics: std::nullopt means no signedness semantics; true means signed integer type; false means unsigned integer type. More...
 
std::optional< unsigned > getHashIdentifierNumber () const
 Given a hash_identifier token like #123, try to parse the number out of the identifier, returning std::nullopt if it is a named identifier like #x or if the integer doesn't fit. More...
 
std::string getStringValue () const
 Given a token containing a string literal, return its value, including removing the quote characters and unescaping the contents of the string. More...
 
std::optional< std::string > getHexStringValue () const
 Given a token containing a hex string literal, return its value or std::nullopt if the token does not contain a valid hex string. More...
 
std::string getSymbolReference () const
 Given a token containing a symbol reference, return the unescaped string value. More...
 
SMLoc getLoc () const
 
SMLoc getEndLoc () const
 
SMRange getLocRange () const
 

Static Public Member Functions

static std::optional< uint64_t > getUInt64IntegerValue (StringRef spelling)
 For an integer token, return its value as an uint64_t. More...
 
static StringRef getTokenSpelling (Kind kind)
 Given a punctuation or keyword token kind, return the spelling of the token as a string. More...
 

Detailed Description

This represents a token in the MLIR syntax.

Definition at line 20 of file Token.h.

Member Enumeration Documentation

◆ Kind

Definition at line 22 of file Token.h.

Constructor & Destructor Documentation

◆ Token()

mlir::Token::Token ( Kind  kind,
StringRef  spelling 
)
inline

Definition at line 31 of file Token.h.

Member Function Documentation

◆ getEndLoc()

SMLoc Token::getEndLoc ( ) const

Definition at line 26 of file Token.cpp.

Referenced by getLocRange(), and mlir::detail::Parser::parseDialectSymbolBody().

◆ getFloatingPointValue()

std::optional< double > Token::getFloatingPointValue ( ) const

For a floatliteral token, return its value as a double.

For a floatliteral, return its value as a double.

Returns std::nullopt in the case of underflow or overflow.

Return std::nullopt if the value underflows or overflows.

Definition at line 56 of file Token.cpp.

Referenced by mlir::detail::AsmParserImpl< BaseT >::parseFloat(), and mlir::detail::Parser::parseFloatAttr().

◆ getHashIdentifierNumber()

std::optional< unsigned > Token::getHashIdentifierNumber ( ) const

Given a hash_identifier token like #123, try to parse the number out of the identifier, returning std::nullopt if it is a named identifier like #x or if the integer doesn't fit.

Definition at line 166 of file Token.cpp.

References getKind().

◆ getHexStringValue()

std::optional< std::string > Token::getHexStringValue ( ) const

Given a token containing a hex string literal, return its value or std::nullopt if the token does not contain a valid hex string.

A hex string literal is a string starting with 0x and only containing hex digits.

Definition at line 135 of file Token.cpp.

References getKind(), and getSpelling().

Referenced by parseElementAttrHexValues().

◆ getIntTypeBitwidth()

std::optional< unsigned > Token::getIntTypeBitwidth ( ) const

For an inttype token, return its bitwidth.

Definition at line 64 of file Token.cpp.

References getKind().

Referenced by mlir::detail::Parser::parseNonFunctionType().

◆ getIntTypeSignedness()

std::optional< bool > Token::getIntTypeSignedness ( ) const

For an inttype token, return its signedness semantics: std::nullopt means no signedness semantics; true means signed integer type; false means unsigned integer type.

Definition at line 73 of file Token.cpp.

References getKind().

◆ getKind()

Kind mlir::Token::getKind ( ) const
inline

◆ getLoc()

SMLoc Token::getLoc ( ) const

◆ getLocRange()

SMRange Token::getLocRange ( ) const

◆ getSpelling()

StringRef mlir::Token::getSpelling ( ) const
inline

◆ getStringValue()

std::string Token::getStringValue ( ) const

Given a token containing a string literal, return its value, including removing the quote characters and unescaping the contents of the string.

The lexer has already verified that this token is valid.

Definition at line 86 of file Token.cpp.

References getKind(), and getSpelling().

Referenced by getSymbolReference(), mlir::detail::Parser::parseAttribute(), mlir::detail::Parser::parseAttributeDict(), mlir::detail::Parser::parseNameOrFileLineColLocation(), and mlir::detail::AsmParserImpl< BaseT >::parseOptionalString().

◆ getSymbolReference()

std::string Token::getSymbolReference ( ) const

Given a token containing a symbol reference, return the unescaped string value.

Definition at line 153 of file Token.cpp.

References getSpelling(), getStringValue(), and is().

Referenced by mlir::detail::Parser::parseAttribute(), and mlir::detail::AsmParserImpl< BaseT >::parseOptionalSymbolName().

◆ getTokenSpelling()

StringRef Token::getTokenSpelling ( Kind  kind)
static

Given a punctuation or keyword token kind, return the spelling of the token as a string.

Warning: This will abort on markers, identifiers and literal tokens since they have no fixed spelling.

Definition at line 177 of file Token.cpp.

Referenced by mlir::detail::Parser::parseCommaSeparatedListUntil().

◆ getUInt64IntegerValue() [1/2]

std::optional<uint64_t> mlir::Token::getUInt64IntegerValue ( ) const
inline

Definition at line 83 of file Token.h.

References getSpelling().

◆ getUInt64IntegerValue() [2/2]

std::optional< uint64_t > Token::getUInt64IntegerValue ( StringRef  spelling)
static

For an integer token, return its value as an uint64_t.

For an integer token, return its value as a uint64_t.

If it doesn't fit, return std::nullopt.

Definition at line 45 of file Token.cpp.

Referenced by mlir::detail::Parser::parseDistinctAttr(), mlir::detail::Parser::parseIntegerInDimensionList(), and mlir::detail::Parser::parseStridedLayoutAttr().

◆ getUnsignedIntegerValue()

std::optional< unsigned > Token::getUnsignedIntegerValue ( ) const

For an integer token, return its value as an unsigned.

If it doesn't fit, return std::nullopt.

Definition at line 34 of file Token.cpp.

Referenced by mlir::detail::Parser::parseNameOrFileLineColLocation().

◆ is()

bool mlir::Token::is ( Kind  k) const
inline

◆ isAny() [1/2]

bool mlir::Token::isAny ( Kind  k1,
Kind  k2 
) const
inline

◆ isAny() [2/2]

template<typename... T>
bool mlir::Token::isAny ( Kind  k1,
Kind  k2,
Kind  k3,
T...  others 
) const
inline

Return true if this token is one of the specified kinds.

Definition at line 44 of file Token.h.

References is(), and isAny().

◆ isCodeCompletion()

bool mlir::Token::isCodeCompletion ( ) const
inline

◆ isCodeCompletionFor()

bool Token::isCodeCompletionFor ( Kind  kind) const

Returns true if the current token represents a code completion for the "normal" token type.

Definition at line 203 of file Token.cpp.

Referenced by isOrIsCodeCompletionFor().

◆ isKeyword()

bool Token::isKeyword ( ) const

Return true if this is one of the keyword token kinds (e.g. kw_if).

Definition at line 192 of file Token.cpp.

Referenced by mlir::detail::Parser::isCurrentTokenAKeyword(), isIdentifier(), and mlir::detail::Parser::parseAttributeDict().

◆ isNot() [1/2]

bool mlir::Token::isNot ( Kind  k) const
inline

◆ isNot() [2/2]

template<typename... T>
bool mlir::Token::isNot ( Kind  k1,
Kind  k2,
T...  others 
) const
inline

Return true if this token isn't one of the specified kinds.

Definition at line 54 of file Token.h.

References isAny().

◆ isOrIsCodeCompletionFor()

bool mlir::Token::isOrIsCodeCompletionFor ( Kind  kind) const
inline

Returns true if the current token is the given type, or represents a code completion for that type.

Definition at line 70 of file Token.h.

References is(), and isCodeCompletionFor().


The documentation for this class was generated from the following files: