freedreno,turnip: Add tooling to edit command streams and use them in 'replay' (!19444) · Merge requests · Mesa

Due to an influx of spam, we have had to impose restrictions on new accounts. Please see this wiki page for instructions on how to get full permissions. Sorry for the inconvenience.

Update #1:

Due to abusive users subverting our CI facilities to mine cryptocurrency, breaking out of the container sandbox in the process, we have been forced to take actions to limit the usage of the public runners to official projects only.

The policy will be enforced on 2023-03-23 (or before if we detect abuses).

Please see this issue for more context and to see if and how you are impacted.

freedreno,turnip: Add tooling to edit command streams and use them in 'replay'

In !19171 (merged) there was a first half of the puzzle, here is the second one.

I chose a similar path to what @krh did in https://gitlab.freedesktop.org/krh/fdtrace. Though instead of decompiling all cmdstreams (and creating a standalone replayer), which could be too much for a big traces, I chose to narrow decompiling and editing to a single cmdstream (which could be picked).

So the idea is:

Find cmdstream in .rd that you want to edit, in next steps it would be identified by its number (cmdstream[%csnumber%])
rddecompiler -s %csnumber% trace.rd > generate_rd.c
Edit the code
Compile it using the example meson.build which uses Mesa as a subproject
replay trace.rd --override=%csnumber% --generator=/path/to/generate_rd

Example of generated code: Danil/mesa$7150

Current limitations:

No CP_LOAD_STATE6 editing;
packet's values are in numeric format, the decoded values are in the comments.

WIP because I there is probably some cleaning to do.

Edited 4 months ago by Danylo Piliaiev

Merge request pipeline #746043 passed for af79e631 3 months ago

Approval is optional

Merged by 3 months ago (Nov 23, 2022 8:21pm UTC) 3 months ago

Merge details

Changes merged into main with af79e631.
Deleted the source branch.

Pipeline #746048 waiting for manual action for af79e631 on main

Activity

3bee8e31 - meson: do not use source_root() when possible

6253a212 - freedreno/decode: Add 'rddecompiler' tool

Compare with previous version
I'm also thinking about being able to override cmdstream during normal driver execution.

Why? Current way to override cmdstreams addresses only hangs, it cannot help with CTS failures and app misrenderings. Adding support for a7xx would likely introduce a lot of those, so it would be great to have a way to easily tweak command streams for those cases.

Which cases we are interested in:

Tests - (same test + same driver) should always produce the same cmdstream, so it doesn't matter how we would identify it the cmdstream;

Full API traces (gfxreconstruct) - should always produce the same cmdtraces;

RenderDoc - possible to inspect every call, would produce unique cmdstreams in no particular order during inspection;
Ideally I'd want to be able to edit just a specific segment, replacing it in all cmdtraces.

As a compromise I think we could identify the cmdstream by its hash (and append the hash while decoding the stream). So it would work in a similar way to have shader substitutions are done.

Thoughts?

This is out of scope of this MR.
341e1b8d - freedreno/decode: Add shader disasm/asm support to rddecompiler
Compare with previous version
Added support for editing shaders:

I cheated a bit and used GCC extension in order to use raw string literals.

Added example meson.build for decompiled source, see rdcompiler-meson.build

I think that's all features I wanted in the scope of this MR.

Edited by Danylo Piliaiev 4 months ago

nice.. IMO no problems for using a gcc extension.

Please register or sign in to reply
b3666fbb - freedreno/decode: Move rd section parsing to a common header

0f4bc465 - freedreno/decode: Print cmdstream idx

212e93c4 - freedreno/replay: Add option to override selected cmdstream

539a5ef0 - meson: do not use source_root() when possible

d55c7e7b - freedreno/decode: Add 'rddecompiler' tool

41efed6c - freedreno/rddecompiler: Add shader disasm/asm support

5141ad41 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
Danylo Piliaiev changed title from WIP: freedreno,turnip: Add tooling to edit command streams and use them in 'replay' to freedreno,turnip: Add tooling to edit command streams and use them in 'replay' 4 months ago
Rob Clark @robclark started a thread on an outdated change in commit 212e93c4 4 months ago

Resolved by Danylo Piliaiev 3 months ago
Rob Clark @robclark started a thread on an outdated change in commit d55c7e7b 4 months ago

Resolved by Danylo Piliaiev 3 months ago
so, a-b on the meson bits, and r-b on the rest. I guess it could use a bit of doc about how the workflow works

Right, I'll add the comment. I'm on vacation till Nov 21st, so I'll do it next week.

Added a how-to to rddecompiler.c.

Please register or sign in to reply
ab189837 - freedreno/replay: Add option to override selected cmdstream

664e1cf1 - meson: do not use source_root() when possible

1e610395 - freedreno/cffdec: Remove dead code

e16adc16 - freedreno/cffdec: Remove packet types

bb25c008 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

7e42e9e6 - freedreno/decode: Add 'rddecompiler' tool

d5cea693 - freedreno/rddecompiler: Add shader disasm/asm support

97a38113 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
97a38113...0fb81352 - 1112 commits from branch mesa:main

008eeecf - freedreno/decode: Move rd section parsing to a common header

ceb570e6 - freedreno/decode: Print cmdstream idx

5696de99 - freedreno/replay: Add option to override selected cmdstream

dd5e45ad - freedreno/cffdec: Remove dead code

44cbbd28 - freedreno/cffdec: Remove packet types

2fefff15 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

67002396 - freedreno/decode: Add 'rddecompiler' tool

46978f32 - freedreno/rddecompiler: Add shader disasm/asm support

0aa1da31 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
0aa1da31...4301f1d8 - 52 commits from branch mesa:main

f97e3663 - freedreno/decode: Move rd section parsing to a common header

07161d98 - freedreno/decode: Print cmdstream idx

c702f823 - freedreno/replay: Add option to override selected cmdstream

fc35c30a - freedreno/cffdec: Remove dead code

4293aa17 - freedreno/cffdec: Remove packet types

ed31854f - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

c06d156d - freedreno/decode: Add 'rddecompiler' tool

ecd55a13 - freedreno/rddecompiler: Add shader disasm/asm support

acf2f020 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
acf2f020...a4690750 - 4 commits from branch mesa:main

bf54a2a1 - freedreno/decode: Move rd section parsing to a common header

66af1123 - freedreno/decode: Print cmdstream idx

6349fc17 - freedreno/replay: Add option to override selected cmdstream

875f6b28 - freedreno/cffdec: Remove dead code

ce643fef - freedreno/cffdec: Remove packet types

4eba8c05 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

e6087211 - freedreno/decode: Add 'rddecompiler' tool

b21aa824 - freedreno/rddecompiler: Add shader disasm/asm support

6a33f347 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
I couldn't merge this branch: CI failed!
f97e3663 - freedreno/decode: Move rd section parsing to a common header

2813f7a0 - freedreno/decode: Print cmdstream idx

61953760 - freedreno/replay: Add option to override selected cmdstream

fb3a56af - freedreno/cffdec: Remove dead code

f9dec9d7 - freedreno/cffdec: Remove packet types

5fe2f891 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

5c124c22 - freedreno/decode: Add 'rddecompiler' tool

57fef513 - freedreno/rddecompiler: Add shader disasm/asm support

4caa8730 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
4caa8730...f0ddb389 - 7 commits from branch mesa:main

52a29dd7 - freedreno/decode: Move rd section parsing to a common header

a2f58340 - freedreno/decode: Print cmdstream idx

34dc57c5 - freedreno/replay: Add option to override selected cmdstream

83e477b8 - freedreno/cffdec: Remove dead code

f24063f4 - freedreno/cffdec: Remove packet types

f722fff3 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

3773c138 - freedreno/decode: Add 'rddecompiler' tool

b53298bf - freedreno/rddecompiler: Add shader disasm/asm support

90a5c4ef - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
I couldn't merge this branch: CI failed!
f97e3663 - freedreno/decode: Move rd section parsing to a common header

2813f7a0 - freedreno/decode: Print cmdstream idx

61953760 - freedreno/replay: Add option to override selected cmdstream

fb3a56af - freedreno/cffdec: Remove dead code

f9dec9d7 - freedreno/cffdec: Remove packet types

5fe2f891 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

39fceb2e - freedreno/decode: Add 'rddecompiler' tool

8393f484 - freedreno/rddecompiler: Add shader disasm/asm support

e2d975cf - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list
e2d975cf...4701508e - 34 commits from branch mesa:main

060e1686 - freedreno/decode: Move rd section parsing to a common header

6d72407d - freedreno/decode: Print cmdstream idx

86e8b933 - freedreno/replay: Add option to override selected cmdstream

00e2f9ef - freedreno/cffdec: Remove dead code

13ab0088 - freedreno/cffdec: Remove packet types

86d27ac4 - freedreno/cffdec: Add type0/type4 vs type3/type7 pkt helpers

03d80e0a - freedreno/decode: Add 'rddecompiler' tool

a7773c3c - freedreno/rddecompiler: Add shader disasm/asm support

af79e631 - freedreno/replay: Add option to replay specific range of cmdstreams

Compare with previous version
Toggle commit list

Please register or sign in to reply

0 Reviewers

Milestone

Time tracking

No estimate or time spent

Lock merge request

Unlocked

4 participants

Reference: mesa/mesa!19444

Source branch: freedreno/feature/cmdstream-editing

freedreno,turnip: Add tooling to edit command streams and use them in 'replay'

freedreno,turnip: Add tooling to edit command streams and use them in 'replay'

Merged by 3 months ago (Nov 23, 2022 8:21pm UTC) 3 months ago

Activity

Recommend

Watch Hyundai's Charging Robot In Action With The IONIQ 6 EV

The Guide to Actionable and Accurate DORA Metrics

To B 战场：没有风口的年代，运营就是风口 | 运营派

比尔盖茨盛赞人工智能：是我一生中唯二的革命性技术进步

You Can Test Google's Bard AI Rival To ChatGPT Now: Here's How

未来学家预测人类将在2030年实现永生

Adobe Launches 'Firefly,' Its Very Own AI Image Generator

Understanding different cache types in Power BI

腾势N7全球首搭法国潮奢科技音响品牌DEVIALET帝瓦雷车载音响

How U.S. Government Agencies Can Fix Their Customer-Service Problem

About Joyk