Problem
The workflow-controller subchart (used by litmus-agent) hardcodes the controller’s log level to debug in the Deployment args. There is no way to override it via values.yaml, so operators cannot reduce log verbosity without forking the chart or using post-rendering (e.g. Helm post-renderer).
Relevant code:
The chart does not expose this in workflow-controller/values.yaml, so the level cannot be customized.
Impact: log cost, mainly in multi-cluster environments
The workload litmus-agent-workflow-controller is the main source of logs from the agent. With debug fixed and no override:
- In our environment, 20 clusters running the agent generate over 30,000 log records every 15 minutes from this component alone. At
info level this volume would be significantly lower.
- Debug logs multiply ingestion, storage, and retention cost across logging pipelines (e.g. Datadog, Elastic, CloudWatch).
- In production, many operators want
info or warn by default and use debug only for troubleshooting. Without the ability to override via values, we are forced to maintain a Helm post-renderer or a fork.
Offering a values-based override would let operators align log level with their cost and operational policies without patching the chart.
Proposed solution
-
Add an optional value in workflow-controller/values.yaml (e.g. under appSettings), defaulting to debug to preserve current behavior and compatibility:
appSettings:
configmapName: "workflow-controller-configmap"
executorImage: "litmuschaos/argoexec:v3.3.1"
containerRuntimeExecutor: "k8sapi"
loglevel: "debug"
-
Use it in the Deployment template instead of the hardcoded literal:
- --loglevel
- {{ .Values.appSettings.loglevel | default "debug" }}
-
Keep default debug so existing installs and defaults stay unchanged; operators who need less noise can set e.g. workflow-controller.appSettings.loglevel: "info" in their values.
Workaround (current)
Until the chart supports this, we use a Helm post-renderer to patch the rendered manifest and change debug to info. It works but is brittle (depends on exact template output) and we would prefer to configure it via values.
Thank you for considering this; it would help a lot for multi-cluster and production deployments.
Problem
The
workflow-controllersubchart (used bylitmus-agent) hardcodes the controller’s log level todebugin the Deployment args. There is no way to override it viavalues.yaml, so operators cannot reduce log verbosity without forking the chart or using post-rendering (e.g. Helm post-renderer).Relevant code:
litmus-helm/charts/litmus-agent/charts/workflow-controller/templates/deployment.yaml
Line 47 in 89f39c3
The chart does not expose this in
workflow-controller/values.yaml, so the level cannot be customized.Impact: log cost, mainly in multi-cluster environments
The workload
litmus-agent-workflow-controlleris the main source of logs from the agent. Withdebugfixed and no override:infolevel this volume would be significantly lower.infoorwarnby default and usedebugonly for troubleshooting. Without the ability to override via values, we are forced to maintain a Helm post-renderer or a fork.Offering a values-based override would let operators align log level with their cost and operational policies without patching the chart.
Proposed solution
Add an optional value in
workflow-controller/values.yaml(e.g. underappSettings), defaulting todebugto preserve current behavior and compatibility:Use it in the Deployment template instead of the hardcoded literal:
Keep default
debugso existing installs and defaults stay unchanged; operators who need less noise can set e.g.workflow-controller.appSettings.loglevel: "info"in their values.Workaround (current)
Until the chart supports this, we use a Helm post-renderer to patch the rendered manifest and change
debugtoinfo. It works but is brittle (depends on exact template output) and we would prefer to configure it via values.Thank you for considering this; it would help a lot for multi-cluster and production deployments.