slim containers, local or ocrd_network #468

bertsky · 2025-07-04T14:54:45Z

builds upon both #465 and #449, merging both conceptually

idea is to still rely on Makefile as much as possible, allow user to switch between client (ocrd_network) and standalone (local Docker) behaviour for processors, but run-network configuration: only optional and less complicated

I therefore also removed the processors definitions from the example configs, as they are no longer needed – you can still reduce the enabled/active list via normal make OCRD_MODULES=....

Example usage:

build/pull images, install executables

As usual, do

make all

If you want to disable some modules, or only enable a few, use OCRD_MODULES= and DISABLED_MODULES= as usual. This can still be done via local.mk. For example, do

echo OCRD_MODULES="core ocrd_tesserocr ocrd_wrap workflow_configuration" >> local.mk
make all

run standalone

If you did not generate a network setup so far (or already ran network-clean afterwards), processors should just spin up the local containers, as in #346.

ocrd-preprocess-image -J

If you want to pass specific options to docker run, then use DOCKER_RUN_OPTS. For example, if the default CWD=/data is not the best for you, consider:

DOCKER_RUN_OPTS="-v /some/path:/data" ocrd-preprocess-image -m my-workspace/mets.xml -I OCR-D-IMG -O OCR-D-BIN ...

Notably, you can still use the ocrd multi-CLI binary here. Each container has its own alias, just prefixing the image e.g.

ocrd-wrap-ocrd # will be `ocrd` inside ocrd/wrap image
ocrd-tesserocr-ocrd # will be `ocrd` inside ocrd/tesserocr image

For example:

ocrd-tesserocr-ocrd resmgr download ocrd-tesserocr-recognize frak2021.traineddata

run network

Just running…

make network-setup

…will create a docker-compose.yml for all installed processors (along with the Processing Server etc), along with a .env (to be customised) and ocrd-processing-server-config.yaml.

If you want to use ODEM config file (with fewer processors), pass that:

make network-setup OCRD_NETWORK_CONFIG=run-network/odem-workflow-config.yaml

To then start all services, do

make network-start
# or, equivalently
docker compose up --wait -d

If you were using a configuration file without preconfigured processors, then the typical ocrd_all definitions of profiles are usable with the standard Docker Compose mechanism for profiles, e.g.

make network-start DOCKER_PROFILES=minimum
# or, equivalently
DOCKER_PROFILES=minimum docker compose up --wait -d
# or, equivalently
docker compose --profile minimum up --wait -d

[+] Running 38/38
 ✔ Container ocrd-rabbitmq                  Running                                                                                                                                                                                                                                   0.0s 
 ✔ Container ocrd-mongodb                   Running                                                                                                                                                                                                                                   0.0s 
 ✔ Container ocrd-page-transform            Started                                                                                                                                                                                                                                  23.9s 
 ✔ Container ocrd-processing-server         Started                                                                                                                                                                                                                                   1.8s 
 ✔ Container ocrd-tesserocr-fontshape       Started                                                                                                                                                                                                                                   8.7s 
 ✔ Container ocrd-cis-ocropy-denoise        Started                                                                                                                                                                                                                                   1.8s 
 ✔ Container ocrd-cis-ocropy-deskew         Started                                                                                                                                                                                                                                  14.1s 
 ✔ Container ocrd-skimage-binarize          Started                                                                                                                                                                                                                                  14.2s 
 ✔ Container ocrd-cis-ocropy-dewarp         Started                                                                                                                                                                                                                                   3.9s 
 ✔ Container ocrd-cis-ocropy-resegment      Started                                                                                                                                                                                                                                  15.7s 
 ✔ Container ocrd-cis-align                 Started                                                                                                                                                                                                                                  11.6s 
 ✔ Container ocrd-tesserocr-segment-table   Started                                                                                                                                                                                                                                   6.1s 
 ✔ Container ocrd-tesserocr-segment-region  Started                                                                                                                                                                                                                                   5.7s 
 ✔ Container ocrd-skimage-denoise           Started                                                                                                                                                                                                                                   6.6s 
 ✔ Container ocrd-cis-ocropy-clip           Started                                                                                                                                                                                                                                  17.6s 
 ✔ Container ocrd-tesserocr-crop            Started                                                                                                                                                                                                                                   6.0s 
 ✔ Container ocrd-preprocess-image          Started                                                                                                                                                                                                                                   2.5s 
 ✔ Container ocrd-cis-ocropy-train          Started                                                                                                                                                                                                                                  11.5s 
 ✔ Container ocrd-tesserocr-binarize        Started                                                                                                                                                                                                                                   1.4s 
 ✔ Container ocrd-tesserocr-segment-line    Started                                                                                                                                                                                                                                  18.5s 
 ✔ Container ocrd-dummy                     Started                                                                                                                                                                                                                                   8.2s 
 ✔ Container ocrd-olahd-client              Started                                                                                                                                                                                                                                  18.8s 
 ✔ Container ocrd-tesserocr-recognize       Started                                                                                                                                                                                                                                   2.2s 
 ✔ Container ocrd-keraslm-rate              Started                                                                                                                                                                                                                                  11.1s 
 ✔ Container ocrd-cis-postcorrect           Started                                                                                                                                                                                                                                  18.5s 
 ✔ Container ocrd-tesserocr-deskew          Started                                                                                                                                                                                                                                   7.4s 
 ✔ Container ocrd-tesserocr-segment-word    Started                                                                                                                                                                                                                                   7.5s 
 ✔ Container ocrd-cis-ocropy-binarize       Started                                                                                                                                                                                                                                  17.1s 
 ✔ Container ocrd-page2alto-transform       Started                                                                                                                                                                                                                                  16.7s 
 ✔ Container ocrd-fileformat-transform      Started                                                                                                                                                                                                                                  14.1s 
 ✔ Container ocrd-filter                    Started                                                                                                                                                                                                                                   1.4s 
 ✔ Container ocrd-skimage-normalize         Started                                                                                                                                                                                                                                  14.1s 
 ✔ Container ocrd-cis-ocropy-segment        Started                                                                                                                                                                                                                                  11.0s 
 ✔ Container ocrd-tesserocr-segment         Started                                                                                                                                                                                                                                  18.4s 
 ✔ Container ocrd-cis-ocropy-recognize      Started                                                                                                                                                                                                                                  17.9s 
 ✔ Container ocrd-altotopdf                 Started                                                                                                                                                                                                                                  12.8s 
 ✔ Container ocrd-skimage-denoise-raw       Started                                                                                                                                                                                                                                  12.6s 
 ✔ Container ocrd-pagetopdf                 Started

Then any processor will normally talk to the Processing Server:

ocrd-preprocess-image --help

using Processing Server at localhost:8000
Usage: ocrd-preprocess-image [OPTIONS]

Options:
  -m, --mets TEXT                 [required]
  -I, --input-file-grp TEXT
  -O, --output-file-grp TEXT
  -g, --page-id TEXT
  -p, --parameter TEXT            Parameters, either JSON string or path to
                                  JSON file
  -P, --parameter-override TEXT...
                                  Parameter override
  --result-queue-name TEXT
  --callback-url TEXT
  --agent-type TEXT
  --help                          Show this message and exit.

If you don't want that, for example because you need to know the installed resources, you can use the override variable:

DOCKER_RUN_POLICY=local ocrd-preprocess-image -L

/usr/local/lib/python3.8/site-packages/ocrd_wrap/param_im6convert-denoise-raw.json
/usr/local/lib/python3.8/site-packages/ocrd_wrap/param_scribo-cli-binarize-sauvola-ms-split.json

To stop the services, don't forget to do

make network-stop
# or, equivalently
docker compose down

examples of generated files:

ocrd-all-images.yaml


ocrd/core
ocrd/kraken
ocrd/detectron2
ocrd/page2alto
ocrd/cor-asv-ann
ocrd/keraslm
ocrd/wrap
ocrd/fileformat
ocrd/olena
ocrd/dinglehopper
ocrd/docstruct
ocrd/nmalign
ocrd/segment
ocrd/tesserocr
ocrd/cis
ocrd/pagetopdf
ocrd/calamari
ocrd/anybaseocr
ocrd/froc
ocrd/doxa
ocrd/eynollah
ocrd/olahd-client
ocrd/workflow-configuration

.env

OCRD_PS_MTU=1300

OCRD_PS_PORT=8000

MONGODB_USER=admin

MONGODB_PASS=admin

MONGODB_URL=mongodb://${MONGODB_USER}:${MONGODB_PASS}@ocrd-mongodb:27017

RABBITMQ_USER=admin

RABBITMQ_PASS=admin

RABBITMQ_URL=amqp://${RABBITMQ_USER}:${RABBITMQ_PASS}@ocrd-rabbitmq:5672

USER_ID=1000

GROUP_ID=1000

DATA_DIR=/tmp/mydata

RES_VOL=ocrd-models

INTERNAL_CALLBACK_URL=http://ocrd-processing-server:${OCRD_PS_PORT}

RES_VOL=ocrd-models

USER_ID=1003

GROUP_ID=100

note: the latter overrides the previous settings for USER ID etc

ocrd/eynollah

pulled: 2025-07-04T14:34:06+02:00
tools: ["ocrd-eynollah-segment","ocrd-sbb-binarize"]
profiles: maximum

venv/bin/ocrd-eynollah-segment

#!/usr/bin/env python

import sys
import os
import pathlib
import subprocess


import os
import sys
import pathlib

# detect whether to run standalone or in ocrd_network
dotenv = pathlib.Path(".env")
if dotenv.exists():
    from dotenv import dotenv_values
    dotenv = dotenv_values(dotenv)
    ps_port = dotenv.get('OCRD_PS_PORT', '')
    if ps_port and ps_port.isdecimal():
        pass
        print("using Processing Server at localhost:%s" % ps_port, file=sys.stderr)
    else:
        ps_port = ''
        print("no OCRD_PS_PORT found in .env - starting local container", file=sys.stderr)
else:
    ps_port = ''
    print("no .env found - starting local container", file=sys.stderr)

# allow overriding detection
if policy := os.environ.get('DOCKER_RUN_POLICY', None):
    if policy in ['ocrd_network', 'client'] and not ps_port:
        exit("cannot apply DOCKER_RUN_POLICY=" + policy + ": needs OCRD_PS_PORT via .env")
    if policy in ['local', 'standalone'] and ps_port:
        ps_port = ''
        print("DOCKER_RUN_POLICY overrides: starting local container", file=sys.stderr)


if not ps_port:
    # avoid re-interpreting by shell
    args = ['docker', 'run', '--rm']
    args.extend('-v ocrd-models:/usr/local/share/ocrd-resources -u 1003'.split())
    args.extend(os.environ.get('DOCKER_RUN_OPTS', '').split())
    # try to be smart: bind-mount CWD as /data if not mounted yet
    datapath = None
    for arg in args:
        if arg.endswith(':/data'):
            datapath = arg[:arg.index(':')]
            break
        if arg.endswith(',destination=/data'):
            typ, src, dst = arg.split(',')
            datapath = src.replace('source=', '')
            break
    if not datapath:
        datapath = str(pathlib.Path().absolute())
        args.extend(['-v', datapath + ':/data'])
    if datapath.endswith('/'):
        datapath = datapath[:-1]
    args.append('ocrd/eynollah')
    args.append('ocrd-eynollah-segment')
    # try to be smart: translate host to container data paths
    args.extend([arg.replace(datapath + '/', '/data/') if arg.startswith('/') else arg
                 for arg in sys.argv[1:]])
    ret = subprocess.run(args)
    sys.exit(ret.returncode)

import click
from ocrd_network.cli import client_cli

run_cli = client_cli.commands['processing'].commands['run']


def callback(*args, **kwargs):
    kwargs['address'] = "http://localhost:" + ps_port
    kwargs['block'] = True
    kwargs['print_state'] = True
    return run_cli.callback("ocrd-eynollah-segment", *args, **kwargs)


params = [param for param in run_cli.params
          if param.name not in [
                  'processor_name',
                  'address',
                  'block',
                  'print_state',
          ]]
cli = click.Command(name="ocrd-eynollah-segment",
                    callback=callback,
                    params=params,
                    help=run_cli.help)

if __name__ == "__main__":
    cli()

venv/bin/ocrd-process

#!/usr/bin/env python


import os
import pathlib

# detect whether to run standalone or in ocrd_network
dotenv = pathlib.Path(".env")
if dotenv.exists():
    from dotenv import dotenv_values
    dotenv = dotenv_values(dotenv)
    ps_port = dotenv.get('OCRD_PS_PORT', '')
    if ps_port and ps_port.isdecimal():
        pass
        print("using Processing Server at localhost:%s" % ps_port)
    else:
        ps_port = ''
        print("no OCRD_PS_PORT found in .env - starting local container")
else:
    ps_port = ''
    print("no .env found - starting local container")

# allow overriding detection
if policy := os.environ.get('DOCKER_RUN_POLICY', None):
    if policy in ['ocrd_network', 'client'] and not ps_port:
        exit("cannot apply DOCKER_RUN_POLICY=" + policy + ": needs OCRD_PS_PORT via .env")
    if policy in ['local', 'standalone'] and ps_port:
        ps_port = ''
        print("DOCKER_RUN_POLICY overrides: starting local container")


if not ps_port:
    from ocrd.cli.process import process_cli


    if __name__ == "__main__":
        process_cli()

from ocrd_network.cli import client_cli
import click


run_cli = client_cli.commands['workflow'].commands['run']

def callback(*args, **kwargs):
    kwargs['address'] = "http://localhost:" + ps_port
    kwargs['block'] = True
    kwargs['print_state'] = True
    return run_cli.callback(*args, **kwargs)

params = [param for param in run_cli.params
          if param.name not in [
                  'address',
                  'block',
                  'print_state',
          ]]
cli = click.Command(name="ocrd-process", callback=callback, params=params, help=run_cli.help)

if __name__ == "__main__":
    cli()

docker-compose.yml

networks:
  default:
    driver: bridge
    driver_opts:
      com.docker.network.driver.mtu: ${OCRD_PS_MTU}

services:
  ocrd-processing-server:
    container_name: ocrd-processing-server
    image: ocrd/core:latest
    environment:
      - MONGODB_USER=${MONGODB_USER:-admin}
      - MONGODB_PASS=${MONGODB_PASS:-admin}
      - RABBITMQ_USER=${RABBITMQ_USER:-admin}
      - RABBITMQ_PASS=${RABBITMQ_PASS:-admin}
      - OCRD_NETWORK_SOCKETS_ROOT_DIR=${SOCKETS_DIR:-/data/sockets}
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources
    command: ocrd network processing-server -a 0.0.0.0:8000 /data/ocrd-processing-server-config.yaml
    healthcheck:
      test: ["CMD", "curl", "-f", "http://0.0.0.0:8000"]
      interval: 60s
      timeout: 10s
      start_period: 30s
      retries: 2
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - ocrd-resources:/usr/local/share/ocrd-resources
      - "${DATA_DIR}:/data"
      - "${PWD}/ocrd-all-tool.json:/build/core/src/ocrd/ocrd-all-tool.json"
      - "${PWD}/ocrd-processing-server-config.yaml:/data/ocrd-processing-server-config.yaml"
    ports:
      - ${OCRD_PS_PORT}:8000

  ocrd-mongodb:
    container_name: ocrd-mongodb
    image: mongo:latest
    environment:
      - MONGO_INITDB_ROOT_USERNAME=${MONGODB_USER:-admin}
      - MONGO_INITDB_ROOT_PASSWORD=${MONGODB_PASS:-admin}
    ports:
      - "27018:27017"
    healthcheck:
      test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
      interval: 60s
      timeout: 10s
      start_period: 10s
      retries: 3

  ocrd-rabbitmq:
    container_name: ocrd-rabbitmq
    image: rabbitmq:3-management
    environment:
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_USER:-admin}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASS:-admin}
    ports:
      - "5672:5672"
      - "15672:15672"
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
      interval: 60s
      timeout: 10s
      start_period: 10s
      retries: 3

  ocrd-dummy:
    image: ocrd/core
    container_name: ocrd-dummy
    command: ocrd-dummy worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: []
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

  ocrd-filter:
    image: ocrd/core
    container_name: ocrd-filter
    command: ocrd-filter worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: []
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

  ocrd-kraken-binarize:
    image: ocrd/kraken
    container_name: ocrd-kraken-binarize
    command: ocrd-kraken-binarize worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: [maximum]
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

  ocrd-kraken-recognize:
    image: ocrd/kraken
    container_name: ocrd-kraken-recognize
    command: ocrd-kraken-recognize worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: [maximum]
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

 ocrd-kraken-segment:
    image: ocrd/kraken
    container_name: ocrd-kraken-segment
    command: ocrd-kraken-segment worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: [maximum]
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

  ocrd-detectron2-segment:
    image: ocrd/detectron2
    container_name: ocrd-detectron2-segment
    command: ocrd-detectron2-segment worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: [maximum]
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

  ocrd-page2alto-transform:
    image: ocrd/page2alto
    container_name: ocrd-page2alto-transform
    command: ocrd-page2alto-transform worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: []
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

...

  ocrd-page-transform:
    image: ocrd/workflow-configuration
    container_name: ocrd-page-transform
    command: ocrd-page-transform worker --database $MONGODB_URL --queue $RABBITMQ_URL
    profiles: []
    depends_on: 
      - ocrd-mongodb
      - ocrd-rabbitmq
      - ocrd-processing-server
    user: "${USER_ID}:${GROUP_ID}"
    volumes:
      - "${DATA_DIR}:/data"
      - ocrd-resources:/usr/local/share/ocrd-resources
    environment:
      - OCRD_NETWORK_LOGS_ROOT_DIR=${LOGS_DIR:-/data/logs}
      - XDG_CONFIG_HOME=/usr/local/share/ocrd-resources

volumes:
  ocrd-resources:
    external: true
    name: ${RES_VOL}

Co-authored-by: Stefan Weil <sw@weilnetz.de>

logging hack no longer needed with slim containers

Co-authored-by: Robert Sachunsky <38561704+bertsky@users.noreply.github.com>

…and fat-container Docker builds

- Config: remove `.dest` and `.dest_env` - put generated files into default locations (`docker-compose.yml` and `.env`) - Config: remove `.run_network_dir` - - Config: rename `environment.data_dir_host` to `data_dir` - Config: add `environment.res_vol` (named volume for resources) - Config.from_file: if no `processors` are configured, then read from makefile-generated `ocrd-all-images.yaml` etc. - VOLUMES_TEMPLATE: set `ocrd-resources.name` dynamically in `.env` - PROCESSING_SERVER_TEMPLATE: do not generate PS config file via service YAML, but introduce PROCESSING_SERVER_CONFIG_TEMPLATE w/ command `create-psconfig` - PROCESSING_SERVER_TEMPLATE: add healthcheck - RABBITMQ_TEMPLATE: add healthcheck - MONGODB_TEMPLATE: add healthcheck - DELEGATOR_PROCESSOR_TEMPLATE: differentiate runtime behaviour between ocrd_network/client and standalone/local: - if `.env` exists, try to parse it for `OCRD_PS_PORT` - if `$DOCKER_RUN_POLICY` is set, then override `.env` detection - if PS port is usable (and not overridden), behave as client - otherwise behave as standalone Docker run for that image and executable - in the latter case, if no `/data` volume has yet been mounted via `DOCKER_RUN_OPTS` at build time (make variable) or at run time (shell variable), then bind-mount the CWD - make all (`delegate_docker`): build executable scripts via `create-client` in rules, instead of config-driven general `create-clients` - covers non-processors as well, and covers all processors for all enabled modules (not just those enabled in network configuration) - make all/images (`pullpolicy_docker`): add filesystem artifacts for Docker images, so - images will not have to be re-built/pulled each time, but are timestamped - we can also aggregate metadata (executable names, Docker profiles) for `network-setup` - make ocrd-all-images.yaml: just aggregates all such artifact names - each image rule: add `DOCKER_PROFILES` target-specific variable - make docker-compose.yml: depend on `ocrd-all-images.yaml` etc. - make .env: after `create-dotenv`, customize DOCKER_VOL_NAME and UID/GID - make network-start, make network-stop: no dependency on config file here (i.e. just an alias for standard `docker compose up --wait -d` / `down`; `wait_for_startup` obsolete because of standard compose `healthcheck`)

bertsky · 2025-07-04T15:03:36Z

make ocrd-process work again
extend ocrd-process to cover the standalone case, i.e. translate each processor with docker run
ensure non-processor executables always behave standalone (never try to be clients for ocrd or some other CLI)
get rid of unnecessary native installation bits (deps-ubuntu, ocrd-all-tool.json etc)
re-think CI and CD
update makefile self-documentation (help, comments, ...)
update readme
update changelog
write some primer for the release notes

- print diagnostics to stderr to avoid mixing with actual output - in standalone mode, be careful not to re-expand arguments via shell - in standalone mode, try to replace host-side with container-side path arguments

- in client mode, delegate to `ocrd network client workflow run` - in standalone mode, delegate to 'ocrd process' (which in turn will translate processor calls to standalone Docker run via installed processor delegators)

… for non-processors

joschrew

For me this works now, maybe some smaller parts could be optimized but generally I'd merge this

Makefile

run-network/ocrd-all-config.yaml

Makefile

Co-authored-by: joschrew <91774427+joschrew@users.noreply.github.com>

Makefile

Co-authored-by: Konstantin Baierer <kba@users.noreply.github.com>

joschrew and others added 20 commits August 20, 2024 16:11

Add run network sample

45df6fe

Fix typo in run-network/creator.py

a31568d

Co-authored-by: Stefan Weil <sw@weilnetz.de>

Format run-network/creator.py

ed0f4ca

Adapt run-network to slim containers

2ccfecf

Update run-network example config

219552f

logging hack no longer needed with slim containers

Create a venv in run-network as a client

8b63cf8

Update run-network: add parameter_override option

ab0f660

Improve run-network creator and make-task

d3fa81f

Split network-setup make task

b9d0f84

Co-authored-by: Robert Sachunsky <38561704+bertsky@users.noreply.github.com>

Use format-str only in run-network-templates

283e0ba

Add minimum medium maximum profiles to run-network

5739560

Add ocrd-all-tool.json to run-network temporarily

1fd919f

Use docker-compose var for run-network profiles

e40acac

Add type-hints to run-network-creator

924ac78

Add volume for resources to run-network

f76921f

Use external volume for run-network

e8d9c6a

delegate to slim-container Docker images, remove native installation …

0acbfa1

…and fat-container Docker builds

adapt ocrd-all-*.json rules, separate init-vol-models from images / all

d32441c

Merge remote-tracking branch 'origin/run-network' into slim-cli

5bc2eb0

bertsky requested review from kba and joschrew July 4, 2025 14:57

Robert Sachunsky added 4 commits July 6, 2025 02:25

processor delegator: improve…

4339693

- print diagnostics to stderr to avoid mixing with actual output - in standalone mode, be careful not to re-expand arguments via shell - in standalone mode, try to replace host-side with container-side path arguments

create delegator for ocrd-process analogously…

5f66216

- in client mode, delegate to `ocrd network client workflow run` - in standalone mode, delegate to 'ocrd process' (which in turn will translate processor calls to standalone Docker run via installed processor delegators)

delegators: only attempt .env detection for processors - always local…

10fe0d2

… for non-processors

fix 2 typos

04aac32

joschrew approved these changes Jul 14, 2025

View reviewed changes

Makefile Outdated Show resolved Hide resolved

run-network/ocrd-all-config.yaml Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

Makefile Show resolved Hide resolved

bertsky and others added 2 commits July 14, 2025 17:04

fix typo in DOCKER_PULL_POLICY

60f5fb3

Co-authored-by: joschrew <91774427+joschrew@users.noreply.github.com>

network-clean: nofail on docker volume rm

4ac3f5a

Co-authored-by: joschrew <91774427+joschrew@users.noreply.github.com>

run-network/ocrd-all-config.yaml: use CWD /data by default

6be0065

Co-authored-by: joschrew <91774427+joschrew@users.noreply.github.com>

This was referenced Aug 28, 2025

Improve ocrd-network-client error handling OCR-D/core#1337

Merged

Remove processor server OCR-D/spec#265

Merged

kba reviewed Sep 10, 2025

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

Makefile Show resolved Hide resolved

Robert Sachunsky and others added 7 commits September 23, 2025 14:48

make clean independent of images-clean

25e1f18

make help: improve

f29c74b

separate clean-vol-models off network-clean

e46044e

help: add network-related targets

904ebf9

network-setup: default to $PWD as /data volume

1be67aa

fix pyyaml dependency

3379d4d

Co-authored-by: Konstantin Baierer <kba@users.noreply.github.com>

images: replace jq with python call

6e1a9d0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

slim containers, local or ocrd_network #468

slim containers, local or ocrd_network #468

Uh oh!

bertsky commented Jul 4, 2025 •

edited

Loading

Uh oh!

bertsky commented Jul 4, 2025 •

edited

Loading

Uh oh!

joschrew left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slim containers, local or ocrd_network #468

Are you sure you want to change the base?

slim containers, local or ocrd_network #468

Uh oh!

Conversation

bertsky commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

build/pull images, install executables

run standalone

run network

examples of generated files:

Uh oh!

bertsky commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joschrew left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bertsky commented Jul 4, 2025 •

edited

Loading

bertsky commented Jul 4, 2025 •

edited

Loading