[FR] Try to parse strings as integers when requiring an integer

[pindab0ter]

Is your feature request related to a problem? Please describe.
In an attempt to set up different values for different environments (e.g. staging and production) I'm passing the values as environment variables and using them like so:

services:
  projectname:
    container_name: app
    image: ${IMAGE_ID}
    deploy:
      replicas: ${ECS_REPLICAS}

However, this results in the following error:

Traceback (most recent call last):
  File "/home/runner/work/projectname/venv/bin/ecs-compose-x", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/work/projectname/venv/lib/python3.12/site-packages/ecs_composex/cli.py", line 194, in main
    settings = ComposeXSettings(**vars(args))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/projectname/venv/lib/python3.12/site-packages/ecs_composex/common/settings.py", line 177, in __init__
    self.set_content(kwargs, content)
  File "/home/runner/work/projectname/venv/lib/python3.12/site-packages/ecs_composex/common/settings.py", line 525, in set_content
    content_def = ComposeDefinition(files, content)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/projectname/venv/lib/python3.12/site-packages/compose_x_render/compose_x_render.py", line 324, in __init__
    jsonschema.validate(
  File "/home/runner/work/projectname/venv/lib/python3.12/site-packages/jsonschema/validators.py", line 1332, in validate
    raise error
jsonschema.exceptions.ValidationError: '1' is not of type 'integer'

Failed validating 'type' in schema['properties']['services']['patternProperties']['^[a-zA-Z0-9._-]+$']['properties']['deploy']['properties']['replicas']:
    {'type': 'integer'}

On instance['services']['projectname']['deploy']['replicas']:
    '1'

Describe the solution you'd like
When requiring an integer, I would like a string to attempt a string to int conversion before discarding the value for not being the correct type.

Describe alternatives you've considered
YAML does not support conditionals, so the values will have to be set outside of the YAML file. When bringing them in they are always treated as strings, for as far as I know. I do not know of a way to mitigate this.

[JohnPreston]

Hello @pindab0ter
The error comes from the fact that the JSON schema for this only accepts integer whereas you have set ${ECS_REPLICAS} and unfortunately, env vars == string, that's why it's upset about it.

[JohnPreston]

Seeing https://github.com/compose-spec/compose-spec/blob/main/schema/compose-spec.json#L521 the definition was updated since so I will update that to accept a string as argument :)

[pindab0ter]

I have found !!int. Unfortunately, the type coercion happens before the environment variable substitution happens, which means that this is not a solution.

I look forward to the update with support for both strings and ints in those fields.

We're also using x-scaling with ScheduledActions:

    x-scaling:
      Range: "${ECS_OFFICE_HOURS_MIN_CAPACITY}-${ECS_OFFICE_HOURS_MAX_CAPACITY}"
      TargetScaling:
        CpuTarget: 50
      ScheduledActions:
        - Timezone: Europe/Amsterdam
          Schedule: cron(30 7 ? * MON-FRI)
          ScheduledActionName: Scale.Up
          ScalableTargetAction:
            MinCapacity: "${ECS_OFFICE_HOURS_MIN_CAPACITY}"
            MaxCapacity: "${ECS_OFFICE_HOURS_MAX_CAPACITY}"
          MacroParameters:
            AddServiceName: true
        - Timezone: Europe/Amsterdam
          Schedule: cron(30 17 ? * MON-FRI)
          ScheduledActionName: Scale.Down
          ScalableTargetAction:
            MinCapacity: "${ECS_AFTER_HOURS_MIN_CAPACITY}"
            MaxCapacity: "${ECS_AFTER_HOURS_MAX_CAPACITY}"

I could not find these in the schema you linked. Do you reckon this will work, too?

[JohnPreston]

Yeah okay, so lots of these values need to be casted to int() from str in case you use env vars, I see. I don't use env vars all that much in my worflows and I guess defo not for numbers 🤣

Should be easy to do though

[pindab0ter]

In the meantime, a workaround is to use envsubst to generate the file from a template.

[JohnPreston]

In the meantime, a workaround is to use envsubst to generate the file from a template.

That's a clever one. I did have a FR (to myself 🤣 ) to allow having the compose files be a template itself using Jinja2 syntax.
I guess I could put this one back on the top of the list and have something that would be

    x-scaling:
      Range: "${ECS_OFFICE_HOURS_MIN_CAPACITY}-${ECS_OFFICE_HOURS_MAX_CAPACITY}"
      TargetScaling:
        CpuTarget: 50
      ScheduledActions:
        - Timezone: Europe/Amsterdam
          Schedule: cron(30 7 ? * MON-FRI)
          ScheduledActionName: Scale.Up
          ScalableTargetAction:
            MinCapacity: {{ env(ECS_OFFICE_HOURS_MIN_CAPACITY) | int }}
            MaxCapacity: {{ env(ECS_OFFICE_HOURS_MAX_CAPACITY') | int }}
          MacroParameters:
            AddServiceName: true
        - Timezone: Europe/Amsterdam
          Schedule: cron(30 17 ? * MON-FRI)
          ScheduledActionName: Scale.Down
          ScalableTargetAction:
            MinCapacity: {{ env('ECS_AFTER_HOURS_MIN_CAPACITY' | int }}
            MaxCapacity: {{ env('ECS_AFTER_HOURS_MAX_CAPACITY') | int }}

And render that before processing the files. Thoughts @pindab0ter ?
It has the benefit to have the `%{{ raw }} % marker if one wants these ${} to remain that way and not changed.

[pindab0ter]

That looks like it would work! I suppose it would have to be announced clearly in the documentation that the file is interpreted as a Jinja template, or maybe there could be a command flag to make use of that.

It has the benefit to have the `%{{ raw }} % marker if one wants these ${} to remain that way and not changed.

With envsubst you can use $${ENV_VARIABLE} if you want it to end up as ${ENV_VARIABLE} after substitution. I have also not had any issues with types, as MinCapacity: ${ECS_OFFICE_HOURS_MIN_CAPACITY} just becomes MinCapacity: 1.

[JohnPreston]

There would be a command line option indeed to instruct to use Jinja2 template functions/filters first indeed.
Envsubst is very sweet but looking at what can be done better possibly.

[pindab0ter]

Looking forward to see what you come up with!