General OpPackage Central Migration Guidance

1) Avoid usage of the ultra-generic Tensor type when possible. - When used for an input tensor, it assumes your implementation can handle ALL datatypes, layouts, and memory locations - When used for an output tensor, it assumes the tensor ought to be in the Flat layout and located in Main Memory (not TCM) - Ex: If you know your tensor ought to only ever contain Int32s and be in Crouton format, just use Int32CroutonTensor

2) Related to (1), try to enumerate ALL scenarios your implementation supports, or extend the implementation - Even if you would prefer an implementation run exclusively in the fastest/lowest-footprint scenarios (TCM, Flat):

  • Maximize flexibility for the central layout and placement pass to work with

  • The central pass does bias toward making your impls run as fast as possible

  • Will make it more likely that you don’t experience failures due to exceeding memory thresholds

  • You can adjust the DEF_TENSOR_PROPERTIES to your liking after enumeration to reflect your preferences

3) Remove any additions/transformations of formatting/memory placement ops in favor of DEF_TENSOR_PROPERTIES where you can - We provide suggestions on where you can move such rules for our system to try and maximize compatibility, BUT… - In the fullness of time you shouldn’t write optimizations containing:

  • ForceFormat_Flat

  • ForceFormat_Crouton

  • flat_from_vtcm

  • flat_to_vtcm

  • crouton_from_vtcm

  • crouton_to_vtcm

  • ConvLayer.opt.activations_to_vtcm

  • ConvLayer.opt.activations_from_vtcm

  • You should have a DEF_TENSOR_PROPERTIES written for any Op you define that might exist at the end of LAYOUT_AND_PLACEMENT

  • Failure to specify an ops tensor properties will lead to us assuming all tensors are Flat + MainMemory

4) Please use DEF_PACKAGE_OPTIMIZATION, not DEF_OPT for your rewrite rules. - To maximize compatibility, we like to know when optimizations originate from an external OpPackage vs. internal optimizations

Do not worry if it still puzzles you, we will annouce a very helpful script tool to help you with this migration