Optimization Grammar

Each of the macros described in the previous section must conform to the grammar below. It must be possible to compile the rules using C++ compiler, and parse the rules using an external tool. So, certain constructs (such as (int)n rather than int(n)) will work in C++ but are not permitted by the rules.

Lexical

The only separators used in the grammar: Left and Right parentheses, and the comma (also, the plus sign is used in only one context):

'('  ')'  ','   '+'

All other characters must be one of the following:

<string_constant>           "Chars"

Nothing unprintable, no space or escapes; so " and \ cannot be used. Empty string is not allowed. As in C/C++, if two or more string constants appear with nothing (or just white space) separating them, they are concatenated. However, full C escapes are allowed for strings used in MESSAGE, etc.

<integer_const>            E.g. 1230u

As in C, including with U or L suffixes; and with optional sign. Use of an octal value >=8 (e.g. 010) will generate a warning.

Suffix ‘L’ is ignored; ‘U’ causes the value to be seen as ‘size_t’ rather than ‘int’}

<float_const>             E.g.  -1.331e-2

As in C, and with optional sign; ‘f’ suffix allowed.

<name>         [A-Za-z_][A-Za-z_0-9]*

Any C identifier (or keyword). The names OK, true, false are recognized as bool constants. Also, INF and NEG_INF are recognized as float constants representing infinity’ and -infinity. Throughout the grammar, quoted symbols such as 'ADD' appear, this means “a which is ADD”.

<dtype_tag>               DType::<name>

This represents a constant of type DType; the must be one of the tags of the DType enum.

General

operand_tag ::=
   <string_constant>      // representing an operand tag

opstring ::=
   <string_constant>      // representing the function of an Op node to be matched or built

Top Level

For the external rule-parsing tool, it is required that the DEF_PACKAGE_OPTIMIZATION and the following ‘(’ both appear at the start of the same source line; otherwise, tokens may be separated by arbitrary white space, and/or C++ comments.

Anything after the closing ‘)’ of a DEF_PACKAGE_OPTIMIZATION(...), up to the end of file or the next DEF_PACKAGE_OPTIMIZATION, is ignored; however, the first non-space character following a DEF_PACKAGE_OPTIMIZATION(...) may not be a left-parenthesis or comma.

optimization_rule ::=
   DEF_PACKAGE_OPTIMIZATION ( 'pass_spec' , 'match_expression ',' constraint_expression ',' replacement_rule ')'

pass_spec ::=
       <name>
    |  <name> '+' <integer_const>

Note: in this context, * the must be one of the predefined names for pass groups * the integer constant may only be a decimal constant, with no sign or suffix

With similar set of rules, user can set the configure a custom Op with cost and flag as the following:

optimization_rule ::=
   DEF_PACKAGE_OP_AND_COST_AND_FLAGS( 'function_name', 'op_name_as_str', 'cost_of_an_op', 'resources_utilized')

resources_utilized::=
    IS_CONST            // consant propogation (default flag value)
|   RESOURCE_HVX,       // utilizing HVX hardware

* User can set more than one flag, separated by comma.

To simply register an Op, without any cost or resources, use the following:

DEF_PACKAGE_OP( 'function_name' , 'op_name_as_str')

Match

See also ref: OptMatch

match_expression ::=
    match_op1

Note that * Op( opstr, inp1, inp2,.. ) matches a specific graph op with the given list of 0 or more input operands * OpVarIn( opstr, inp1, inp2... ) matches a specific graph op with the given list of 0 or more input operands and possibly additional input operands. * If the same operand tag appears more than once in the match expression, it indicates that these parts of the matched pattern must all reference the same node. * LET( "opname", ... ) can be wrapped around any Op or OpVarIn (other than the root), to attach an operand tag to the output of that Op. * Within a given match expression, a tag used as the first parameter of a LET cannot appear anywhere within the second operand of the same LET, and may not be used as the first parameter of any other LET.

match_op1 ::=
       'Op'      '(' opstring [',' match_op]* ')'
    |  'OpVarIn' '(' opstring [',' match_op]* ')'

match_op2 ::=
       'LET' '(' operand_tag ',' match_op1 ')'
    |  match_op1

match_op ::=
       operand_tag
   |  match_op2

Constraint

Note: * All of the operand_tag in constraints must exist in the corresponding ‘match’ rule, or must be the special tag "*". * in general, constraint expressions have types: - bool, int, size, float, dtype

• Currently, ‘dtype’ support is limited to:

  • constants e.g. DType::Float

  • comparison to dtype using EQ and NE only;

  • SELECT( bool, dtype, dtype ) -> dtype

  • property DTYPE_OF(param) gives a dtype

  • conversion to int in via INT()

  • WITH_OUTPUT_TYPE(..) requires a dtype expression as its first parameter.

For more detail on the constraint functions, see ref: OptConstraint

constraint_expression ::=
       cst_expr                // must be bool type

cst_expr ::=
       cst_const
    |  cst_option
    |  cst_oper
    |  cst_property
    |  cst_constval
    |  cst_external
    |  cst_macro

cst_const ::=
       'OK' |  'true' |  'false'         // type = bool
    |  'INF' | 'NEG_INF'                 // type = float
    |  <integer_const>                   // type = 'int'; or size, if U suffix
    |  <float_const>                     // type = 'float'
    |  <dtype_tag>                       // type = 'dtype'
    | 'LAYOUT_CHUNKSIZE' '(' <name>  ',' <integer_const> ')'  // type = size
            // the <name> is the name of a tensor class.

cst_option ::=
       'OPTION_INT' '(' "name_of_option" ')'         // type = int
    |  'OPTION_UINT' '(' "name_of_option" ')'        // type = size
    |  'OPTION_BOOL' '(' "name_of_option" ')'        // type = bool
    |  'OPTION_FLOAT' '(' "name_of_option" ')'       // type = float

Read values from graph options. Options can be read as any type; the value is converted (perhaps with loss). A ‘bool’ option is read as integer 0 or 1; a ‘string’ option reads as 0 if empty, 1 if not. OPTION_BOOL will convert a non-bool type as if by (value!=0).

cst_oper ::=
        typecast '('  cst_expr ')'
    |   'NOT' '(' cst_expr ')'                    // bool -> bool
    |   'NEG' '(' cst_expr ')'
    |   'ABS' '(' cst_expr ')'
    |   'IS_POW2' '(' cst_int ')'                // int, or size -> bool
    |   'SELECT' '('   cst_expr ',' cst_expr  ',' cst_expr ')' // first must be bool
    |   'ROUNDUP' '('  cst_int ',' cst_int ')'      // int/size only; second must be power of 2
    |   reduce_op '('  cst_expr [ ',' cst_expr]* ')'
    |   binary_op '('  cst_expr ',' cst_expr ')'
    |   compare_op '(' cst_expr ',' cst_expr ')'        // result is bool

typecast ::=
    'UINT' |  'INT' |  'FLOAT' | 'DTYPE'

reduce_op ::=
        'AND' |  'OR'  |   'XOR'
    |   'ADD' |  'MUL'
    |   'MIN' |  'MAX'

Note, REM(a,b) and MOD(a,b) are the same when applied to values >0; for negative values, REM(a,b), if not zero, has the same sign as ‘a’; MOD(a,b), if not zero, has the same sign as ‘b’.

binary_op ::=
       'SUB' |  'DIV' | 'REM' | 'MOD'

compare_op ::=
        'EQ' |  'NE'
     |  'LT' |  'GT' |  'LE' |  'GE'

cst_opref ::=
        operand_tag
    |   'INPUT_OF'  '(' cst_opref ',' cst_int ')'
    |   'OUTPUT_OF'  '(' cst_opref ',' cst_int ')'
    |   'SELECT' '(' cst_expr ',' cst_opref ',' cst_opref ')'  // expr must be bool

The “cst_property” operations below extract properties of the output of the operand specified by the cst_opref.

cst_property ::=
        'RANK_OF' '(' cst_opref ')'
    |   'DIM_OF'  '(' cst_opref ',' cst_int ')'
    |   'STEPSIZE_OF' '(' cst_opref  ')'
    |   'DTYPE_OF' '(' cst_opref  ')'
    |   'ELEMENTSIZE_OF' '(' cst_opref  ')'
    |   'INPUTS_OF' '(' cst_opref ')'
    |   'OUTPUTS_OF' '(' cst_opref ')'

The operations below give a boolean result and can be used to compare two operands. SAME_ENCODING means the operands have the same DType; and if the DType is quantized, the two also have the same quantization.

SAME_OP means that both operands refer to the same node in the graph. It is possible for the operands to refer to different nodes which will later be merged as common sub-expressions; in which case this will return ‘false’

cst_op_compare ::=
        'SAME_ENCODING' '(' cst_opref ',' cst_opref ')'
    |   'SAME_OP'  '(' cst_opref ',' cst_opref ')'

These operations can extract a scalar value from a Const operand at the given index. In case of failure, i.e. when the the operand is not ‘Const’, or when the index is out of range, CONSTVAL_INT returns MIN_INT, and CONSTVAL_FLOAT returns NaN. CONSTVAL_INT will also fail if the value is not an integer that fits in int32. The corresponding CONSTVAL_INT_VALID and CONSTVAL\_FLOAT\_VALID return ‘true’ if the operation will succeed, and ‘false’ if it will not.

cst_constval ::=
        'CONSTVAL_INT'         '(' cst_opref ',' cst_int ')'
    |   'CONSTVAL_INT_VALID'   '(' cst_opref ',' cst_int ')'
    |   'CONSTVAL_FLOAT'       '(' cst_opref ',' cst_int ')'
    |   'CONSTVAL_FLOAT_VALID' '(' cst_opref ',' cst_int ')'

In the grammar, ‘cst_int’ generally means a ‘cst_expr’ of integer or size type. But, when the expression appears within the last parameter of an AUTOSPLIT or OP_ITER , the special operations listed below may also appear; these obtain one of the variables from the iteration.

cst_int ::=
         cst_expr                // with int or size type
    |   'SPLIT_DIM' '(' split_tag ')'
    |   'SPLIT_START' '(' split_tag ')'
    |   'SPLIT_SIZE' '(' split_tag ')'
    |   'ITER_VAR' '(' split_tag ')'   // synonym for SPLIT_START (use with OP_ITER, INPUT_OF, OUTPUT_OF)

EXTERNAL_CONSTRAINT calls an external C++ function which is expected to return bool. The first parameter is an “OperandTag”; the remainder (if any) must evaluate to a scalar type (int, size_t, float, bool) matching the parameter types of the function. You can use general cst_expr, e.g. EXTERNAL_CONSTRAINT( funcname, "Operand", RANK_OF("Operand"))

cst_external ::=
    'EXTERNAL_CONSTRAINT' '(' <name> ','  operand_tag [ ',' cst_expr ]* ')'

‘cst_macro’ are equivalent to the expansions given:

cst_macro ::=
       'IS_QUINT8' '(' cst_opref ')'         // IS_QUINT8(x) => EQ(DTYPE_OF(x),DType::QUInt8)
    |  'IS_QINT8' '(' cst_opref ')'          // IS_QINT8(x) => EQ(DTYPE_OF(x),DType::QInt8)
    |  'IS_QUINT16' '(' cst_opref ')'        // IS_QUINT16(x) => EQ(DTYPE_OF(x),DType::QUInt16)
    |  'IS_QINT16' '(' cst_opref ')'         // IS_QINT16(x) => EQ(DTYPE_OF(x),DType::QInt16)
    |  'IS_QINT32' '(' cst_opref ')'         // IS_QINT32(x) => EQ(DTYPE_OF(x),DType::QInt32)
    |  'IS_INT32' '(' cst_opref ')'          // IS_INT32(x) => EQ(DTYPE_OF(x),DType::Int32)
    |  'IS_FLOAT16' '(' cst_opref ')'        // IS_FLOAT16(x) => EQ(DTYPE_OF(x),DType::Float16)
    |  'IS_FLOAT32' '(' cst_opref ')'        // IS_FLOAT32(x) => EQ(DTYPE_OF(x),DType::Float32)
    |  'IS_FLOAT' '(' cst_opref ')'          // IS_FLOAT(x) => IS_FLOAT32(x)
    |  'DIM_BATCHES' '(' cst_opref ')'       // DIM_BATCHES(x) => DIM_OF(x,0)
    |  'DIM_HEIGHT' '(' cst_opref ')'        // DIM_HEIGHT(x) => DIM_OF(x,1)
    |  'DIM_WIDTH' '(' cst_opref ')'         // DIM_WIDTH(x) => DIM_OF(x,2)
    |  'DIM_DEPTH' '(' cst_opref ')'         // DIM_DEPTH(x) => DIM_OF(x,3)
    |  'DIM_FILTHEIGHT' '(' cst_opref ')'    // DIM_FILTHEIGHT(x) => DIM_OF(x,0)
    |  'DIM_FILTWIDTH' '(' cst_opref ')'     // DIM_FILTWIDTH(x) => DIM_OF(x,1)
    |  'DIM_FILTDEPTH' '(' cst_opref ')'     // DIM_FILTDEPTH(x) => DIM_OF(x,2)
    |  'DIM_NFILTS' '(' cst_opref ')'        // DIM_NFILTS(x) => DIM_OF(x,3)
    |  'SAME_SHAPE' '(' cst_opref ','  cst_opref ')'
            // SAME_SHAPE(x,y) => AND( EQ( DIM_OF(x,3), DIM_OF(y,3)), EQ( DIM_OF(x,2), DIM_OF(y,2)),
            //                        EQ( DIM_OF(x,1), DIM_OF(y,1)), EQ( DIM_OF(x,0), DIM_OF(y,0)))

Replacement

Notes: * Some of the entities in the Replacement Grammar refer to cst_expr from the Constraint grammar * All of the operand_tag must exist in the corresponding ‘match’ rule, or must be the special tag "*". * Some of the entities have a ‘split_tag’ operand, which names a split context. The scope of these names is the entire replacement rule containing them.

The following apply in a well-formed rule: * Each instance of AUTOSPLIT or OP_ITER in a rule must have a distinct split_tag as its second operand (usually there is at most one, and the tag is "I"). * Every other entity referring to a split_tag must be contained within the last operand of an AUTOSPLIT or OP_ITER entity which has the same split_tag in its second operand. * An AUTOSPLIT must contain, within its third operand, at least one such entity referencing the same split_tag. * more detailed checks on the validity of the overall construct could be defined.

For more detail on the replacement operations, see ref: OptReplacement

split_tag ::=
   <string_constant>      // representing a split context

replacement_rule ::=
    repl_op

Note that ‘Operand’ is redundant : whenever a string constant appears in a ‘repl_op’ context, it is assumed to be an operand tag and ‘Operand’ is applied.

repl_op ::=
         cst_opref
    |    'Operand'  '('  operand_tag ')'
    |    'Op'  '(' opstring [',' repl_op]* ')'
    |    'WrapOp'  '(' opstring ',' repl_op ')'
    |    'WrapOpAlways'  '(' opstring ',' repl_op ')'
    |    'gen_Shape' '(' cst_int [',' cst_int ]* ')'
    |    'gen_ShapeOf' '(' cst_opref ')'
    |    'gen_ConstScalar_f32' '(' cst_expr ')'
    |    'gen_ConstScalar_i32' '(' cst_int ')'
    |    'gen_ConstArr_f32' '(' cst_expr ',' cst_int')      // gen_ConstArr_f32(floatval, n) -> array[1,1,1,n] filled with floatval
    |    'gen_ConstArr_i32' '(' cst_expr ',' cst_int')      // gen_ConstArr_i32(intval, n) -> array[1,1,1,n] filled with intval
    |    'gen_ConstArr_vals_i32' '(' cst_int [',' cst_int]* ')'
                 // gen_ConstArr_vals_i32(x,y,z,..) -> array [1,1,1,n] of int filled with x,y,z ...
    |    'AUTOSPLIT' '(' cst_int ',' split_tag ',' cst_int ',' repl_op ')'
    |    'OP_ITER' '(' repl_op ',' split_tag ',' cst_int ',' cst_int ',' repl_op ')'
    |    'SELECT' '(' cst_expr ',' repl_op ',' repl_op ')'
    |    repl_iterop
    |    repl_modifier
    |    repl_apply
    |    repl_macro

OUTPUT_OF( "operand_tag", int_expr ) can be used, in some contexts, to obtain a specific output of a multi-op output.

WrapOp("opname", some_op), which allows exactly one input to "opname", works by evaluating some_op, and then constructing Op("op_name", some_op) with the same Dtype and output as some_op. The Op Id and split-history are inherited in the usual way; it is generally equivalent to WITH_SAME_OUTPUT(X, Op("opname", X)) but without needing to evaluate X twice. However, if some_op is an "opname" Op, it does not construct a new Op, it evaluates to some_op (on the assumption that Op("op_name", Op("op_name",x)) is equivalent to Op("op_name",x). When this assumption is not true, use WrapOpAlways which has the same behaviour, but will always add the new Op.

repl_op listed here under repl_iterop may only be used within the last parameter of an AUTO_SPLIT or OP_ITER, and must reference the same split_tag.

repl_iterop ::=
        'ITER_INPUT_OF' '(' cst_opref ',' split_tag ')'
    |   'AUTOSPLIT_SHAPEFN_APPLY' '(' <name> ',' split_tag [',' apply_parm ]* ')'

Modifiers:

A modifier sets the attributes of the output tensor of any Op() in its last operand, according to its preceding operands. So, the last operand must be a repl_op which constructs an Op (including, another nested modifier).

For WITH_OUTPUT_TYPE: the cst_expr are dtype, zero_off and stepsizeand must be of type dtype, int, float.ResizeDim changes one dimension to a a given value; the first two params are the dimension number and the value to change to.

repl_modifier ::=
        'WITH_SIZE' '('  repl_op  ','  repl_op ')'
    |   'WITH_TYPE' '('  cst_opref  ','  repl_op ')'
    |   'WITH_SAME_OUTPUT'  '('  cst_opref  ','  repl_op ')'
    |   'WITH_OUTPUT_TYPE' '(' cst_expr ',' cst_expr ',' cst_expr ','  repl_op ')'
    |   'WITH_MULTI_OUT' '(' cst_expr ',' repl_op ')'
    |   'ResizeDim' '('  cst_int ','  cst_int  ',' repl_op ')'
    |   'WITH_SAME_ID' '(' cst_opref  ','  repl_op ')'
    |   'WITH_SPLIT_HISTORY' '(' cst_opref  ',' cst_expr ',' repl_op ')'
    |   'WITH_SPLIT_HISTORY' '(' cst_opref  ',' repl_op ')'

(Currently, when WITH_SPLIT_HISTORY has 3 parameters, the second must be an integer constant)

The operations below call external functions.

EXTERNAL_REPLACE is typically used to implement the entire replacement rule, as EXTERNAL_REPLACE( funcname ); where funcname is a function of signature OpRef function(Replacement & rpx, const OpDef &oldop);‘oldop’ is the OpDef being replaced. If the returned OpRef refers to oldop, the graph is not modified.

SHAPEFN_APPLY is given a function of signature OpRef function(Replacement &rpx, ...args...);where the ‘…args…’ are derived from the ‘apply_parm’. Operand names become OpRef, and cst_expr become scalar expressions of the corresponding type.

AUTOSPLIT_SHAPEFN_APPLY is similar, but is given a function of signature conv_split_start_valid(Replacement &rpx, Split_Context const &splitinfo, ...args...);The ‘splitinfo’ is the context indicated by the ‘split_tag’, second parameter of AUTOSPLIT_SHAPEFN_APPLY

The SHAPEFN_APPLY and AUTOSPLIT_SHAPEFN_APPLY functions nominally return an OpRef representing a newly constructed ‘OpDef_Shape’; they may also return a QuickShape (which is a data structure representing a shape in a more compact way), and the framework will convert that to an OpRef.

External functions may also construct a Const data array, and return its OpRef.

These external functions need a annotation comment in the below format. It is added for the benefit of the external parser.

// :::EXTERNAL_SHAPEFN::: { <return_type> function_name(<input_types>); }

See also:

• ref: ShapeFnApply

• ref: AutoSplitShapeFnApply

repl_apply ::=
        'EXTERNAL_REPLACE' '(' <name> ')'
    |   'SHAPEFN_APPLY' '(' <name> [',' apply_parm ]* ')'

In apply_parm context, a <string_constant> is always a repl_op -> operand_tag-> <string_constant>; other types of constant are cst_exp.

apply_parm ::=
         repl_op
    |    cst_expr

The ‘repl_macro’ are equivalent to expanded expressions, as below.

However, in the case of AUTOSPLIT_SLICE, TYPICAL_SLICE, CHANGEDIM_SLICE, the implementation may construct the first operand only once, and AUTOSPLIT_SLICE may evaluate the ‘size’ input only once. So, these operands should not be an Op().

See also :

• ref:Replacement::AUTOSPLIT_SLICE

• ref:Replacement::TYPICAL_SLICE

• ref:Replacement::CHANGEDIM_SLICE

repl_macro ::=
        'AUTOSPLIT_SLICE' '('  cst_opref ',' repl_op, ',' repl_op ')'
                // AUTOSPLIT_SLICE( in, start,size) =>
                //     WITH_SIZE( size, WITH_TYPE( in, Op("Slice_shape", in, start, size)))
    |   'TYPICAL_SLICE' '(' cst_opref ',' split_tag ')'
                // TYPICAL_SLICE( in, tag )=>
                //     AUTOSPLIT_SLICE( in,
                //           AUTOSPLIT_SHAPEFN_APPLY( simple_split_start, tag, in ),
                //           AUTOSPLIT_SHAPEFN_APPLY( simple_split_size, tag, in ))
    |   'CHANGEDIM_SLICE' '(' cst_opref ',' split_tag ',' new_dim ')'
                // CHANGEDIM_SLICE( in, tag )=>
                //     AUTOSPLIT_SLICE( in,
                //           AUTOSPLIT_SHAPEFN_APPLY( simpledim_split_start, tag, in, new_dim ),
                //           AUTOSPLIT_SHAPEFN_APPLY( simpledim_split_size, tag, in, new_dim ))
    |   'OpMultiOut' '(' cst_expr ',' cst_expr ',' opstring [',' repl_op]* ')'
                // OpMultiOut( n_out, outno, "opstr", .. inputs .. ) =>
                //           Op( "$Out", WITH_MULTI_OUT( n_out, Op( "opstr", .. inputs ..)),
                //               gen_shape(0,0,n_out, outno));

Restrictions :

A Replacement rule should meet the restrictions below, most of which are not reflected in the grammar. Rules which do not conform may still work, but may not continue to work in future implementations.

In the below, “existing opref” means a cst_opref as defined in the grammar; Such an expression always refers to a node which is in the graph before the replacement rule is applied.

• The first operand to WITH_SIZE, WITH_TYPE, WITH_SAME_OUTPUT, WITH_SAME_ID, WITH_SPLIT_HISTORY must not be an Op construction (defined below). It can be an existing opref; or in the case of WITH_SIZE, a shape construction (defined below).

• The operand to gen_ShapeOf should be an existing opref. It can be a shape construction, but then the gen_ShapeOf is redundant.

• The first operand to AUTOSPLIT_SLICE, TYPICAL_SLICE, CHANGEDIM_SLICE must be an existing opref.

• The second and third operands to AUTOSPLIT_SLICE must be existing opref, or shape construction (or a SELECT which yields one of these).

• The last operand to a modifier must be an Op construction.

• The first operand of OP_ITER must either be Op(...), or an existing opref. Note that when it is Op(...), the constructed Op is always temporary, even when the iteration does not add additional inputs.

• The entire replacement rule may not simply be "*"; and it may not be a SELECT or nested SELECT which is capable of evaluating to "*".

It is permitted to be an existing opref other than "*"; or to be a SELECT which may evaluate to an existing opref; in these cases the replacement rule just ‘bypasses’ some existing part of the graph; e.g the output of x -> IntToFloat -> FloatToInt might be bypassed to just "x".

• AUTOSPLIT and EXTERNAL_REPLACE may only appear as the “root” of a replacement rule, or within a SELECT or nested SELECT at the root of a rule.

A ‘‘shape construction’’ is one of :

• gen_Shape, gen_ShapeOf

• SHAPEFN_APPLY or AUTOSPLIT_SHAPEFN_APPLY, for a function which returns a shape

• SELECT where the second and third operands are both shape constructions.

An ‘‘Op Construction’’ is one of :

• Op( ... ) or OpMultiOut

• WrapOp("name", x) or WrapOpAlways("name", x). It doesn’t make sense to wrap these in a modifier other than WITH_SAME_ID or WITH_SPLIT_HISTORY, unless x is an Op construction to which the modifier can apply.

• OP_ITER, AUTOSPLIT_SLICE, TYPICAL_SLICE, CHANGEDIM_SLICE

• Any modifier (which must itself contain an Op construction, as the last operand).

• SELECT where at least one of the second and third operands is an Op construction.

Strictly speaking, AUTOSPLIT and EXTERNAL_REPLACE are Op constructions, but may not be used inside modifiers.

Examples : We provided some common optimization utility functions usage examples, please check here.