xtensa: update to espressif/llvm-project #2533

b1llow · 2024-11-03T18:30:19Z

Your checklist for this pull request

I've documented or updated the documentation of every API function and struct this PR changes.
I've added tests that prove my fix is effective or that my feature works (if possible)

Detailed description

https://github.com/espressif/llvm-project

for llvm-capstone sync with this PR capstone-engine/llvm-capstone#65

Test plan

...

Closing issues

...

Rot127

What a great update! Excellent!

Only small nit picks.

Also, please seriously consider to add the instruction formats as well.
You can easily add them just like PPC does:

Copy function: https://github.com/capstone-engine/llvm-capstone/blob/48774db25fbb9d241bf8ec1d6d3d00be61726c07/llvm/utils/TableGen/PrinterCapstone.cpp#L2714-L2744
Base instruction class seems to be XtensaInst\d\d (depending on bit width. So the class before the XtensaInst\d\d is the format class) (https://github.com/espressif/llvm-project/blob/2db67a2fd77dc134f025cbcd79812762877c44f1/llvm/lib/Target/Xtensa/XtensaInstrFormats.td#L12C7-L26)
Maybe even OR the the instruction width into the format enum:

enum {
  XTENSA_FORMAT_INVALID = 0,
  XTENSA_FORMAT_RRR_INST,
  XTENSA_FORMAT_16BIT = 1 << 16,
  XTENSA_FORMAT_24BIT = 1 << 17,
...
}

This feature is very useful. Because if Capstone decoes stuff incorrectly and users are in a rush, they can always extract operand bits manually. So it provides an easy way of hot fixing stuff and checking for bits.

arch/Xtensa/XtensaInstPrinter.c

Rot127 · 2024-11-04T11:40:21Z

arch/Xtensa/XtensaInstPrinter.c

+		}
+	}
+	}
+	printInstruction(MI, Address, O);


It i probably a good idea to support also alias with printAliasInstr.
But you can also skip it, if you are in a rush. Just please open an issue about it.

I don't feel like it's a particularly important feature, and I think it could be supported later.

Sure, no problem. Please open an issue about it after merge.

suite/auto-sync/src/autosync/cpptranslator/patches/LLVM_DEBUG.py

suite/cstest/include/test_detail_xtensa.h

suite/cstest/src/test_detail_xtensa.c

arch/Xtensa/XtensaMapping.c

tests/MC/Xtensa/l32r.yaml

tests/integration/test_litbase.c

- Delete unused code - Add billow's copyright - Add test for LLVM_DEBUG patch

Rot127 · 2024-11-05T21:13:19Z

In case you missed it: capstone-engine/llvm-capstone#62
Sorry, forgot to merge it before.

Rot127

Nice, almost done. Please just add a detail test case for the: format, mem_base, mem_disp, immediate and access information.

Then we are good to go. Maybe we even get it into the patch release.
@kabeor Please review as well.

b1llow · 2024-11-06T02:12:23Z

In case you missed it: capstone-engine/llvm-capstone#62 Sorry, forgot to merge it before.

https://github.com/espressif/llvm-project/blob/2db67a2fd77dc134f025cbcd79812762877c44f1/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp#L562-L568

This issue has been fixed in the esp branch

Rot127

@kabeor Please take a look :)

XVilka

@kabeor please merge this one before the release

kabeor

Cool! Merge now, thank you.

@Antelox

* Update changelog for V6.0.0-Alpha1 (#2493) * update version to v6-alpha1 * update bindings const values * Update changelog for V6.0.0-Alpha1 * Remove irrelevant changes. (#2495) * Fixing UB santizer, `LITBASE` and assert errors. (#2499) * Update labeler with Xtensa and v6 files. (#2500) * Add hard asserts to all SStream functions and memset MCInst. (#2501) * Only trigger on released action. (#2497) * Fix cstest build with Ninja (#2506) * Tricore EA calculation (#2504) * Update libcyaml dependency in cstest to 1.4.2 (#2508) * AArch64: Replace vararg add_cs_detail by multiple concrete functions Fixes UB caused by various mismatches on how these arguments are passed and read. This became visible when running on PowerPC hosts with e.g. `cstool -d aarch64 204862f8`. Apart from the UB fix, this is meant to be a pure refactor. Partially addresses #2458 * xtensa: Fix Branch Target (#2516) * xtensa: Fix Branch Target * auto-sync: fix byte pattern * xtensa: add branch insn tests * Revert "auto-sync: fix byte pattern" This reverts commit cf8e870. * Fix #2509. (#2510) Compatibility headers should always include the header in the same dir. * Fix stringop-truncation warning some compilers raise. (#2522) * Add CC and VAS compatibility macros (#2525) * Fix endianess issue during assignment. (#2528) * This time actually fix big endian issue. (#2530) * tricore: fixes #2474 (#2523) * tricore: fix auto-sync tricore * tricore: fixes TriCoreGenCSMappingInsnName.inc * tricore: fixes * tricore: try fix ld.a SC * tricore: fixes all * Add TriCore to .github/workflows/auto-sync.yaml * Add TriCore details tests(a15, d15, a10|sp) * Change CI to create Debian Package to Release (#2521) * Updating CI to create Debian package and version is assigned by tag version. Also updating release CI to not use end-of-life workflows * Clear up usage of static libraries. - Python bindings only use the dynamic lib. But built and copied the static ones sometimes nonetheless. - Add toggles to build only static, static/dyn or only dynamic. --------- Co-authored-by: Rot127 <unisono@quyllur.org> * Rename build arguments: (#2534) - BUILD_SHARED_LIBS -> CAPSTONE_BUILD_SHARED_LIBS - BUILD_STATIC_LIBS -> CAPSTONE_BUILD_STATIC_LIBS - BUILD_STATIC_LIBS -> CAPSTONE_BUILD_STATIC_MSVC_RUNTIME * xtensa: update to espressif/llvm-project (#2533) * fix coverity (#2546) - cid 514642 - cid 514643 - cid 514644 - cid 514645 * Move debian package generation to a dispatch only workflow (#2543) * Move deb package gen files int package/deb * Fix basename check * Make debian package generation dispatch only * Python package building rework (#2538) * - Refactored setup.py to remove hacks regarding packaging of wheels for different platforms, improve and cleanup the code - Updated README.txt - Removed old Makefile and build_wheel.sh scripts - Created a new workflow that takes care of building and testing python packages for different platforms/architectures/python versions * Added SPDX headers to the setup.py * - cstest_py: Fixed positional argument since it doesn't accept a `required` flag. It turns to have a mandatory tests folder path - integration_tests.py: Use pathlib to determine the required path - GitHub action: Simplified the tests execution command * GitHub Actions: Run python 3.8 (lowest) and 3.13 (current highest) for native runners only during testings and the rest during tag release * GitHub Action: - Fixed the cibw_build matrix element - Added a step to prepare artifact name * GitHub Action: Added run_tests.py script to run all tests during CI workflow * - Added SPDX headers to the run_tests.py script and to the build-wheels-publish.yml workflow file - Minor fixes to the workflow as pointed out in the PR review - Updated MANIFEST.in to reflect the actual libraries built during python wheel creation process - Use subprocess.run in place of os.system in run_tests.py script * GitHub Action: - Run qemu step only if non-native Linux runner - Added arch:universal2 matrix element for macos-latest runner * Python bindings: Refreshed the list of files needed to be copied for sdist archive * GitHub Action: Commented out arch:x86 matrix elements * GitHub Action: Run qemu step only if non-native Linux runner * GitHub Action: Minor fixes * Python bindings: Added missing .in pattern when collecting src files for sdist archive * Auto-Sync reproducability + ARM update (#2532) * fix xtensa DecodeMR23RegisterClass and add tests for MAC16 instru… (#2551) * fix xtensa `DecodeMR23RegisterClass` and add tests for `MAC16` instructions * revert * Prepare for update (#2552) * Bindings(chore): Fix DeprecationWarning * Version(upgrade): update bindings const * Fix(chore): Fix ARMCC_Invalid is not defined * Update Changelog Version to 6.0.0-Alpha2 (#2553) * Bindings(chore): Fix DeprecationWarning * Version(upgrade): update bindings const * Fix(chore): Fix ARMCC_Invalid is not defined * Changelog: Update to version 6.0.0-Alpha2 --------- Co-authored-by: Rot127 <45763064+Rot127@users.noreply.github.com> Co-authored-by: Florian Märkl <info@florianmaerkl.de> Co-authored-by: billow <billow.fun@gmail.com> Co-authored-by: Andrew <afq2101@columbia.edu> Co-authored-by: Rot127 <unisono@quyllur.org> Co-authored-by: @Antelox <anteloxrce@gmail.com>

@Antelox

* Update changelog for V6.0.0-Alpha1 (capstone-engine#2493) (capstone-engine#2494) * update version to v6-alpha1 * update bindings const values * Update changelog for V6.0.0-Alpha1 * Remove irrelevant changes. (capstone-engine#2496) * Fixing UB santizer, `LITBASE` and assert errors. (capstone-engine#2499) * Update labeler with Xtensa and v6 files. (capstone-engine#2500) * Add hard asserts to all SStream functions and memset MCInst. (capstone-engine#2501) * Only trigger on released action. (capstone-engine#2497) * Fix cstest build with Ninja (capstone-engine#2506) * Tricore EA calculation (capstone-engine#2504) * Update libcyaml dependency in cstest to 1.4.2 (capstone-engine#2508) * AArch64: Replace vararg add_cs_detail by multiple concrete functions Fixes UB caused by various mismatches on how these arguments are passed and read. This became visible when running on PowerPC hosts with e.g. `cstool -d aarch64 204862f8`. Apart from the UB fix, this is meant to be a pure refactor. Partially addresses capstone-engine#2458 * xtensa: Fix Branch Target (capstone-engine#2516) * xtensa: Fix Branch Target * auto-sync: fix byte pattern * xtensa: add branch insn tests * Revert "auto-sync: fix byte pattern" This reverts commit cf8e870. * Fix capstone-engine#2509. (capstone-engine#2510) Compatibility headers should always include the header in the same dir. * Fix stringop-truncation warning some compilers raise. (capstone-engine#2522) * Add CC and VAS compatibility macros (capstone-engine#2525) * Fix endianess issue during assignment. (capstone-engine#2528) * This time actually fix big endian issue. (capstone-engine#2530) * tricore: fixes capstone-engine#2474 (capstone-engine#2523) * tricore: fix auto-sync tricore * tricore: fixes TriCoreGenCSMappingInsnName.inc * tricore: fixes * tricore: try fix ld.a SC * tricore: fixes all * Add TriCore to .github/workflows/auto-sync.yaml * Add TriCore details tests(a15, d15, a10|sp) * Change CI to create Debian Package to Release (capstone-engine#2521) * Updating CI to create Debian package and version is assigned by tag version. Also updating release CI to not use end-of-life workflows * Clear up usage of static libraries. - Python bindings only use the dynamic lib. But built and copied the static ones sometimes nonetheless. - Add toggles to build only static, static/dyn or only dynamic. --------- Co-authored-by: Rot127 <unisono@quyllur.org> * Rename build arguments: (capstone-engine#2534) - BUILD_SHARED_LIBS -> CAPSTONE_BUILD_SHARED_LIBS - BUILD_STATIC_LIBS -> CAPSTONE_BUILD_STATIC_LIBS - BUILD_STATIC_LIBS -> CAPSTONE_BUILD_STATIC_MSVC_RUNTIME * xtensa: update to espressif/llvm-project (capstone-engine#2533) * fix coverity (capstone-engine#2546) - cid 514642 - cid 514643 - cid 514644 - cid 514645 * Move debian package generation to a dispatch only workflow (capstone-engine#2543) * Move deb package gen files int package/deb * Fix basename check * Make debian package generation dispatch only * Python package building rework (capstone-engine#2538) * - Refactored setup.py to remove hacks regarding packaging of wheels for different platforms, improve and cleanup the code - Updated README.txt - Removed old Makefile and build_wheel.sh scripts - Created a new workflow that takes care of building and testing python packages for different platforms/architectures/python versions * Added SPDX headers to the setup.py * - cstest_py: Fixed positional argument since it doesn't accept a `required` flag. It turns to have a mandatory tests folder path - integration_tests.py: Use pathlib to determine the required path - GitHub action: Simplified the tests execution command * GitHub Actions: Run python 3.8 (lowest) and 3.13 (current highest) for native runners only during testings and the rest during tag release * GitHub Action: - Fixed the cibw_build matrix element - Added a step to prepare artifact name * GitHub Action: Added run_tests.py script to run all tests during CI workflow * - Added SPDX headers to the run_tests.py script and to the build-wheels-publish.yml workflow file - Minor fixes to the workflow as pointed out in the PR review - Updated MANIFEST.in to reflect the actual libraries built during python wheel creation process - Use subprocess.run in place of os.system in run_tests.py script * GitHub Action: - Run qemu step only if non-native Linux runner - Added arch:universal2 matrix element for macos-latest runner * Python bindings: Refreshed the list of files needed to be copied for sdist archive * GitHub Action: Commented out arch:x86 matrix elements * GitHub Action: Run qemu step only if non-native Linux runner * GitHub Action: Minor fixes * Python bindings: Added missing .in pattern when collecting src files for sdist archive * Auto-Sync reproducability + ARM update (capstone-engine#2532) * fix xtensa DecodeMR23RegisterClass and add tests for MAC16 instru… (capstone-engine#2551) * fix xtensa `DecodeMR23RegisterClass` and add tests for `MAC16` instructions * revert * Prepare for update (capstone-engine#2552) * Bindings(chore): Fix DeprecationWarning * Version(upgrade): update bindings const * Fix(chore): Fix ARMCC_Invalid is not defined * Update Changelog Version to 6.0.0-Alpha2 (capstone-engine#2553) * Bindings(chore): Fix DeprecationWarning * Version(upgrade): update bindings const * Fix(chore): Fix ARMCC_Invalid is not defined * Changelog: Update to version 6.0.0-Alpha2 --------- Co-authored-by: Wu ChenXu <kabeor00@gmail.com> Co-authored-by: Florian Märkl <info@florianmaerkl.de> Co-authored-by: billow <billow.fun@gmail.com> Co-authored-by: Andrew <afq2101@columbia.edu> Co-authored-by: @Antelox <anteloxrce@gmail.com>

xtensa: update LLVM + and fix

493cdf4

github-actions bot added Auto-Sync-files Auto-Sync LLVM-generated-files Xtensa Arch labels Nov 3, 2024

b1llow added 4 commits November 4, 2024 05:12

xtensa: fix all

c143bc1

xtensa: fix l32 encode and print

1f89e09

xtensa: fix test_litbase.c

67244a8

xtensa: fix details

3adb13e

Rot127 requested changes Nov 4, 2024

View reviewed changes

b1llow added 2 commits November 5, 2024 02:28

xtensa: fix all

f292b38

- Delete unused code - Add billow's copyright - Add test for LLVM_DEBUG patch

xtensa: fix free xtensa detail

7535bde

github-actions bot added the CS-core-files auto-sync label Nov 4, 2024

xtensa: add esp32 options and fix python binding

865293c

github-actions bot added the Python Bindings label Nov 4, 2024

b1llow added 4 commits November 5, 2024 03:35

xtensa: update insn format

4ddd6cc

xtensa: update python binding

9939073

xtensa: add insn format

0661c6c

xtensa: fix fuzz test

bd40188

b1llow marked this pull request as ready for review November 5, 2024 02:25

b1llow requested a review from Rot127 November 5, 2024 02:25

xtensa: fix Xtensa_add_cs_detail_0

4970c4a

Rot127 requested changes Nov 5, 2024

View reviewed changes

b1llow added 3 commits November 6, 2024 10:44

xtensa: add detail test for access and format

d544f4a

xtensa: add detail test for mem and imm

7d5dd26

xtensa: fix python binding

5041ef6

Rot127 approved these changes Nov 6, 2024

View reviewed changes

XVilka approved these changes Nov 8, 2024

View reviewed changes

kabeor approved these changes Nov 10, 2024

View reviewed changes

kabeor merged commit 1ecfb5b into capstone-engine:next Nov 10, 2024
19 checks passed

b1llow deleted the xtensa-espressif branch November 11, 2024 16:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

xtensa: update to espressif/llvm-project #2533

xtensa: update to espressif/llvm-project #2533

Uh oh!

b1llow commented Nov 3, 2024 •

edited

Loading

Uh oh!

Rot127 left a comment •

edited

Loading

Uh oh!

Uh oh!

Rot127 Nov 4, 2024

Uh oh!

b1llow Nov 4, 2024

Uh oh!

Rot127 Nov 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Rot127 commented Nov 5, 2024

Uh oh!

Rot127 left a comment

Uh oh!

b1llow commented Nov 6, 2024

Uh oh!

Rot127 left a comment

Uh oh!

XVilka left a comment

Uh oh!

kabeor left a comment

Uh oh!

Uh oh!

Uh oh!

xtensa: update to espressif/llvm-project #2533

xtensa: update to espressif/llvm-project #2533

Uh oh!

Conversation

b1llow commented Nov 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Rot127 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Rot127 Nov 4, 2024

Choose a reason for hiding this comment

Uh oh!

b1llow Nov 4, 2024

Choose a reason for hiding this comment

Uh oh!

Rot127 Nov 5, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Rot127 commented Nov 5, 2024

Uh oh!

Rot127 left a comment

Choose a reason for hiding this comment

Uh oh!

b1llow commented Nov 6, 2024

Uh oh!

Rot127 left a comment

Choose a reason for hiding this comment

Uh oh!

XVilka left a comment

Choose a reason for hiding this comment

Uh oh!

kabeor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

b1llow commented Nov 3, 2024 •

edited

Loading

Rot127 left a comment •

edited

Loading