-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
1M topics #4341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Why publish batches at 1s interval, why not just nc.Publish() in a for loop as fast as you can? |
I.m simulating here real workflow. Where we have some predefined polling interval. Software grubbing data with this polling interval and need to publish it asap. |
ok, so the server does show 1M msgs/sec which is inline with your expectations, but only until the number of subjects you have published too becomes very high, meaning number of distinct subjects. |
Have you also tried to use more connections on the client? If I understand your test correctly, each instance of |
Do not hold onto no interest subjects from a client in the unlocked cache. If sending lots of different subjects all with no interest performance could be affected. Signed-off-by: Derek Collison <derek@nats.io> Resolves #4341
I see performance improvements in latest version. But not yet target number. Will try more connection per client and will come back. |
Still present, even with higher number of connections. Will continue in slack |
What performance number do you get using e.g. try |
@jnmoyne thanks, localhost, just tried this command |
Going over localhost those are really low numbers...
|
@antlad FWIW: if you constantly push to so many different topics it may be good (performance wise) to disable the sublist cache. On an i5-10210U i get (with cache):
with cache disabled:
Another thing i tried is increasing the default sublistCache size from 1024 to 100000 (slCacheMax constant in the sublist.go file). I don't know the implications but this gives me basically the same performance as when i publish only to 10 subjects:
|
@HeavyHorst Thank you! It works! Ideally will be put this constant into config. |
Recently, doing performance research - we found that after NATS server reaches some large number of topics, its consuming performance degraded. Even if hardware is capable, NATS server is not scaling. Is there any configuration/patch we can do to improve performance with large number of topics? Below are details:
Hardware: Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz (32 cores) . 126GB RAM.


Publishing is done in synthetic test: 1M msg/sec messages, but in 1M topics with pattern
X.X.X
Expected: 1M msg/sec
Actual: msg/sec is around 700k.
NATS server version: 2.9.19
nats-top:
htop:
Test code (compiled with golang 1.20).:
start with
The text was updated successfully, but these errors were encountered: