Skip to content

antlr4: fix rules #7007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

antlr4: fix rules #7007

wants to merge 6 commits into from

Conversation

star-hengxing
Copy link
Contributor

close #6965

end
elseif grammar_name then
table.insert(batchcxx.sourcefiles, path.join(sourcefile_dir, grammar_name .. "Parser.cpp"))
table.insert(batchcxx.sourcefiles, path.join(sourcefile_dir, grammar_name .. "Lexer.cpp"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@waruqi 目前单个 job 里有两个 cxx file 并不会并行编译,这里该怎么优化

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参考的 idl 那个 rule? 感觉你这里参数不对,少了 jobgraph 参数

得参考 https://github.com/xmake-io/xmake/blob/2774ecbc0961e1e2b1f995d8937b44c2c90e11c6/xmake/rules/verilator/verilator.lua#L264

按理里面默认都会并行的。。idl 那个也有问题,估计得改

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idl 那个确实有很多问题,我等会修

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参考的 idl 那个 rule? 感觉你这里参数不对,少了 jobgraph 参数

得参考 https://github.com/xmake-io/xmake/blob/2774ecbc0961e1e2b1f995d8937b44c2c90e11c6/xmake/rules/verilator/verilator.lua#L264

local dependfile = target:objectfile(objectfile)

这行为什么这么写,有点怪怪的

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

忘记了,也可能是 bug 把,记不太清楚了,得调了才知道。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

试了一下,应该直接用 private.action.build.object main 来给每个 cxx 添加 job

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on_build_files 开头用 group 包一层有什么区别么?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on_build_files 开头用 group 包一层有什么区别么?

你可以不包,这不是必须的,如果你想让其他 job 依赖这些 build files 的整体顺序调整,可以 group 下,然后通过一个 jobgraph:add_orders 去调整统一顺序,而不是挨个 build file 一个 job ,去单独调整顺序

Copy link
Member

@waruqi waruqi Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要去参考那个 idl rule ,那个也是人家贡献进来的,很多写的都不对,这些 group 都可以去掉

可以参考 verilator 那个 rule ,就没包

end)
end
end)
end, {jobgraph = true, batch = true, distcc = true})
Copy link
Member

@waruqi waruqi Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里已经用上了 jobgraph 接口,完全不兼容 batch 了,就没必要启用 batch = true 了。。

另外,3.0 都还没 release ,都没 jobgraph 支持,直接改,不会 break 现有用户么?

Copy link
Contributor Author

@star-hengxing star-hengxing Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

所以我下面也写了一个 batchcmd 版的,但好像没有很好的办法让 xmake2 优先选择使用

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个改不了使用顺序

@star-hengxing
Copy link
Contributor Author

star-hengxing commented Apr 28, 2025

还有一个问题,下游 target 依赖 codgen 出来的头文件,设置 set_policy("build.across_targets_in_parallel", false) 没有用。

[ 13%]: compiling.release src\main.cpp
[ 18%]: compiling.g4 src\g4\LuaLexer.g4
[ 22%]: compiling.g4 src\g4\LuaParser.g4
[ 27%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaLexer.cpp
[ 31%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaParser.cpp
[ 36%]: compiling.release src\visitor\Visitor.cpp
[ 40%]: compiling.release src\LuaLexerBase.cpp
error: main.cpp
src\main.cpp(3): fatal error C1083: Cannot open include file: 'LuaLexer.h': No such file or directory

main.cpp 是下游 target 里的文件

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is another problem. The downstream target depends on the header file produced by codgen, and setting set_policy("build.cross_targets_in_parallel", false) is useless.

@waruqi
Copy link
Member

waruqi commented Apr 29, 2025

还有一个问题,下游 target 依赖 codgen 出来的头文件,设置 set_policy("build.across_targets_in_parallel", false) 没有用。

[ 13%]: compiling.release src\main.cpp
[ 18%]: compiling.g4 src\g4\LuaLexer.g4
[ 22%]: compiling.g4 src\g4\LuaParser.g4
[ 27%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaLexer.cpp
[ 31%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaParser.cpp
[ 36%]: compiling.release src\visitor\Visitor.cpp
[ 40%]: compiling.release src\LuaLexerBase.cpp
error: main.cpp
src\main.cpp(3): fatal error C1083: Cannot open include file: 'LuaLexer.h': No such file or directory

main.cpp 是下游 target 里的文件

那是因为,你里面有源码间的依赖。

参考 https://github.com/xmake-io/xmake/blob/dev/tests/projects/other/autogen/autogen_codedep/xmake.lua

codegen 完,直接编译成 .o 插入 objectfiles,而不是生成完,丢给 on_buildfiles 去混入当前编译,缺少源码间的 job 依赖关系,你没指定。

另外,across_targets_in_parallel 废弃了,改用 build.fence,不过目前是兼容的,按理应该可以的。

另外,如果要加源码间以来,可以通过 group name / job name + jobgraph:add_orders 你自己配置上去。。

也可以改用 on_prepare_files 里面去做 codegen ,on_build_files 里面去 build ,就不用配置 jobgraph 顺序,完全隔离的

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is another problem. The downstream target depends on the header file produced by codgen. Setting set_policy("build.cross_targets_in_parallel", false) is useless.

[ 13%]: compiling.release src\main.cpp
[ 18%]: compiling.g4 src\g4\LuaLexer.g4
[ 22%]: compiling.g4 src\g4\LuaParser.g4
[ 27%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaLexer.cpp
[ 31%]: compiling.release build\.gens\antlr4-lua\windows\x64\release\rules\antlr4\src\g4\LuaParser.cpp
[ 36% ]: compiling.release src\visitor\Visitor.cpp
[ 40% ]: compiling.release src\LuaLexerBase.cpp
error: main.cpp
src\main.cpp(3): fatal error C1083: Cannot open include file: 'LuaLexer.h': No such file or directory

main.cpp is a file in the downstream target

That's because you have dependencies between source codes.

Reference https://github.com/xmake-io/xmake/blob/dev/tests/projects/other/autogen/autogen_codedep/xmake.lua

After codegen, directly compile it into .o and insert objectfiles, instead of generating it, throw it to on_buildfiles to mix in the current compilation. It lacks the job dependency between the source code, you did not specify it.

In addition, across_targets_in_parallel is abandoned and used instead with build.fence, but it is currently compatible, which should be OK.

In addition, if you want to add source code, you can configure it yourself through group name / job name + jobgraph:add_orders. .

You can also use on_prepare_files to do codegen and on_build_files to build. You don't need to configure the jobgraph order and are completely isolated.

@waruqi
Copy link
Member

waruqi commented Apr 29, 2025

设置 set_policy("build.across_targets_in_parallel", false) 没有用。

这是因为你 rules 里面没加 add_orders 去配置 rules 间的依赖执行顺序,那么即使设置 build.fence 也没用。

比如:add_orders("current rule name", "c++.build", "c.build")

当然,直接设置 on_prepare_files 会更简单些

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Setting set_policy("build.cross_targets_in_parallel", false) is useless.

This is because you don't add_orders to configure the dependency execution order between rules, so even setting build.fence is useless.

For example: add_orders("current rule name", "c++.build", "c.build")

Of course, it would be easier to set on_prepare_files directly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Antlr4 g4处理有误
3 participants