Gangams/fix cosmic large scale issue

[ganga1980]

This pull request introduces several enhancements and fixes to the Kubernetes API client and related utilities, primarily focusing on improving the robustness and efficiency of resource fetching and parsing from the Kubernetes API. The changes include switching to a streaming JSON parser for large responses, handling gzip-compressed payloads, and adding more resilient error handling and telemetry. Additionally, there are minor improvements to custom metrics availability checks and conditional record processing.

Kubernetes API Client improvements

• Switched the resource fetching logic in KubernetesApiClient#getResourcesAndContinuationTokenV2 to use streaming JSON parsing via Yajl::Parser, with support for gzip-compressed responses for improved performance and lower memory usage on large payloads. Fallback to legacy parsing is retained for error cases.
• Added new dependencies (zlib, stringio, and yajl) to KubernetesApiClient.rb to enable streaming and compressed JSON parsing.

Error handling and telemetry

• Enhanced error handling in the resource fetch logic: logs warnings, sends telemetry for non-success responses, and attempts fallback parsing if streaming fails. Also, comprehensive logging of parse modes, response codes, and performance metrics is included.

Custom metrics and pod inventory

• Improved the custom metrics availability check to handle empty string values for the enable_custom_metrics configuration, making the check more robust.
• Updated in_kube_podinventory.rb to only push pod records for custom metrics when custom metrics are available, preventing unnecessary processing.

[github-actions]

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.