feat: implement distributed rate limiting by redis_rate/v10

[gaius-qi]

Description

This pull request refactors the rate limiting system by replacing the limiters library with redis_rate for distributed rate limiting. It introduces a simplified API for rate limit checks and updates related configurations and middleware. Below are the most important changes grouped by theme:

Dependency Updates:

• Replaced github.com/mennanov/limiters with github.com/go-redis/redis_rate/v10 for distributed rate limiting. (go.mod: [1] [2] [3]
• Removed unused dependencies, including github.com/fatih/color, github.com/hashicorp/consul/api, and others. (go.mod: [1] [2] [3]

Distributed Rate Limiter Refactor:

• Replaced the token bucket implementation with a new Allow method using redis_rate.Limiter. (internal/ratelimiter/distributed_ratelimiter.go: internal/ratelimiter/distributed_ratelimiter.goL21-R55)
• Updated the NewDistributedRateLimiter constructor to accept a rate limit in requests per second. (internal/ratelimiter/distributed_ratelimiter.go: internal/ratelimiter/distributed_ratelimiter.goL21-R55)

Job Rate Limiter Simplification:

• Replaced TakeByClusterID and TakeByClusterIDs with AllowByClusterID and AllowByClusterIDs methods for boolean-based rate limit checks. (internal/ratelimiter/job_ratelimiter.go: [1] [2]
• Updated the refresh method to use the new DistributedRateLimiter. (internal/ratelimiter/job_ratelimiter.go: internal/ratelimiter/job_ratelimiter.goL167-R178)

Configuration Changes:

• Updated DefaultClusterJobRateLimit type to uint32 to align with the new rate limiter API. (manager/config/constants.go: manager/config/constants.goL106-R106)

Middleware Update:

• Replaced the token-based rate limiting logic with the new AllowByClusterIDs method in the job rate limiter middleware. (manager/middlewares/ratelimiter.go: manager/middlewares/ratelimiter.goL37-R37)

Related Issue

Motivation and Context

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation Update (if none of the other choices apply)

Checklist

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 26 lines in your changes missing coverage. Please review.

Project coverage is 34.41%. Comparing base (21255b1) to head (fcb4f4b).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/ratelimiter/job_ratelimiter.go	0.00%	20 Missing ⚠️
internal/ratelimiter/distributed_ratelimiter.go	0.00%	5 Missing ⚠️
manager/middlewares/ratelimiter.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4068      +/-   ##
==========================================
+ Coverage   34.28%   34.41%   +0.12%     
==========================================
  Files         344      346       +2     
  Lines       40380    40458      +78     
==========================================
+ Hits        13846    13922      +76     
- Misses      25617    25622       +5     
+ Partials      917      914       -3     

Flag	Coverage Δ
unittests	34.41% <0.00%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
manager/middlewares/ratelimiter.go	0.00% <0.00%> (ø)
internal/ratelimiter/distributed_ratelimiter.go	0.00% <0.00%> (ø)
internal/ratelimiter/job_ratelimiter.go	0.00% <0.00%> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Liam-Zhao]

LGTM

[chlins]

lgtm