Profile Contract / Deployment Loading [3 days]

[xmonader]

Which package/s are you suggesting this feature for?

Dashboard

Is your feature request related to a problem? Please describe

Too slow loading of contracts/deployments

Describe the solution you'd like

Analyze the performance of contract loading and deployment processes to identify bottlenecks. Research/Develop optimization strategies to improve efficiency and reduce latency.

[samaradel]

Work in Progress (WIP):

• Load all deployments in parallel
• Handle migrations in parallel if needed
• Add grid client cache
• Add batch processing utility
• Create grid clients in parallel with caching
• Load contracts and node IDs in parallel for each machine
• Process consumptions in batches
• Process WireGuard configs in batches

[samaradel]

Work in Progress (WIP):

Implemented processes

• Node info caching
• Batch processing for contract normalization, just like Consumptions and WireGuard
• Parallel data fetching

P.S. For my account (which has 4 contracts), it takes like 3 to 4 seconds to load and calculate the total contract cost. Will test it with many contracts

[samaradel]

Work Completed:

• Implemented the same behaviour for k8s and Caprover
• Contracts loading are tested with 60 contracts in the table, and it still takes 3 to 4 seconds on the first fetch and paginations, but it takes more when selecting from the dropdown to view more pages (10, 20, 50), which is normal

[samaradel]

Work Completed:

Use Promise.allSettled instead of Promise.all

Update:

replace all Promise.all across all deployment files

[samaradel]

Work in Progress (WIP):

• Batch processing and normalization strategy
• Optimize loadContracts function calls
• Add error message for !grid
• Log errors for k8s loading failures
• Fully parallel cluster detail loading

[samaradel]

Work Completed:

• Handle the failure of VM deployments
• Group chunk 3 conditions into a variable for VMs and K8s
• Improve migrateModule calls
• Run batches in parallel
• Handle batch errors if they exist
• Add unit test for batch process function

[samaradel]

Work Completed:

• Remove dead code
• Remove get_nodeInfo_with_cache file and use it as a separate function
• Create a test unit for the getNodeInfoWithCache function
• Create clear cache function
• Extract helper functions to reduce the nesting
• Fix Batch process function usage

[samaradel]

Work Completed:

checked the comments #4233 (comment) and #4233 (comment) and it's all implemented

[samaradel]

Work Completed:

Fix loading network duplicates

[samaradel]

Work in Progress (WIP):

For some reason, the pagination has disappeared from development, and I had it after merging it into my branch
I see now the deployments took a longer time after merging, investigating..