0
基于DL的CostModel
source link: https://zhen8838.github.io/2023/06/16/dl-costmodel/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
调研一些使用机器学习/深度学习方法构造神经网络CostModel的论文.
TLP: A Deep Learning-based Cost Model for Tensor Program Tuning
他这里是把对源代码的schedule的类型进行onehot, 然后名字参数进行 tokenize, 数值参数不改变.
\[ F = F_{1} (\tau) (F_{2} (id) |F_{3} (num)) \\ F_1 : \text{PrimitiveType} \rightarrow \text{OnehotVector} \\ F_2 : \text{NameParam} \rightarrow \text{Token} \\ F_3 : \text{Number} \rightarrow \text{Number} \\ \text{PrimitiveType} \in { \text{split}, \text{reorder}, \text{fuse} } \\ \text{NameParam} := \text{id} \]
特征提取的流程图如下:
他的模型基本上是基于transformer,
讲数据加载进来之后分为input[:setp_size,:feat_size],
这里setp_size,feat_size分别为25,22. 应该说默认一共调度25次,
以及每个调度的参数长22.
Efficient Automatic Scheduling of Imaging and Vision Pipelines for the GPU
这个是通过分析原始调度中的一系列特征值进行分类.
将pipeline_features, schedule_features送到两个输入头中,
然后分别进行全连接之后再concat之后继续全连接.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK