Skip to content

Batch-prediction across multiple GPUs and more efficient patch-prediction #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Batch-prediction across multiple GPUs and more efficient patch-prediction #48

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
b0a8b61
Improved loading of models to allow providing a directory, added a fe…
apacha Aug 22, 2022
4112c6f
Rewrote binarization script to always use patches, but in a much more…
apacha Aug 30, 2022
1baa70e
Improving comments in the code
apacha Aug 30, 2022
1c7e8c5
Improved formatting
apacha Aug 30, 2022
9db2bcc
Remove obsolete comment
apacha Aug 30, 2022
7edf34a
Adding a bit more documentation
apacha Aug 30, 2022
4a97eac
Removed --patches flag, as the new version automatically uses patches…
apacha Sep 15, 2022
9ef8259
Merge branch 'master-upstream'
apacha Nov 22, 2022
cade5dd
reintroduce TF stderr / log silencer
bertsky Feb 14, 2023
342e94e
reinstate ensemble combination and in-memory prediction
bertsky Feb 14, 2023
4086c69
adapt OCR-D wrapper
bertsky Feb 14, 2023
6dc83dd
comment out explicit GC calls / session management
bertsky Feb 14, 2023
3ade5ec
Merge pull request #1 from bertsky/batch-prediction
apacha Mar 19, 2023
4abd5b2
Adding garbage collection and cleanup to avoid memory leaks again.
apacha Mar 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ https://huggingface.co/SBB/sbb_binarization

```sh
sbb_binarize \
-m <path to directory containing model files \
-m <path to directory containing model files> \
<input image> \
<output image>
```
Expand All @@ -40,7 +40,7 @@ Images containing a lot of border noise (black pixels) should be cropped beforeh
### Example

```sh
sbb_binarize -m /path/to/model/ myimage.tif myimage-bin.tif
sbb_binarize -m /path/to/models/ myimage.tif myimage-bin.tif
```

To use the [OCR-D](https://ocr-d.de/) interface:
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ setuptools >= 41
opencv-python-headless
ocrd >= 2.38.0
tensorflow >= 2.4.0
mpire
6 changes: 5 additions & 1 deletion sbb_binarize/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@
"""

from click import command, option, argument, version_option, types

from .sbb_binarize import SbbBinarizer


@command()
@version_option()
@option('--model-dir', '-m', type=types.Path(exists=True, file_okay=False), required=True, help='directory containing models for prediction')
@argument('input_image')
@argument('output_image')
def main(model_dir, input_image, output_image):
SbbBinarizer(model_dir).run(image_path=input_image, save=output_image)
binarizer = SbbBinarizer()
binarizer.load_model(model_dir)
binarizer.binarize_image_file(image_path=input_image, save_path=output_image)
9 changes: 5 additions & 4 deletions sbb_binarize/ocrd_cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ def setup(self):
raise FileNotFoundError("Does not exist or is not a directory: %s" % model_path)
# resolve relative path via OCR-D ResourceManager
model_path = self.resolve_resource(str(model_path))
self.binarizer = SbbBinarizer(model_dir=model_path, logger=LOG)
self.binarizer = SbbBinarizer()
self.binarizer.load_model(model_path)

def process(self):
"""
Expand Down Expand Up @@ -110,7 +111,7 @@ def process(self):

if oplevel == 'page':
LOG.info("Binarizing on 'page' level in page '%s'", page_id)
bin_image = cv2pil(self.binarizer.run(image=pil2cv(page_image)))
bin_image = cv2pil(self.binarizer.binarize_image(pil2cv(page_image)))
# update METS (add the image file):
bin_image_path = self.workspace.save_image_file(bin_image,
file_id + '.IMG-BIN',
Expand All @@ -124,7 +125,7 @@ def process(self):
LOG.warning("Page '%s' contains no text/table regions", page_id)
for region in regions:
region_image, region_xywh = self.workspace.image_from_segment(region, page_image, page_xywh, feature_filter='binarized')
region_image_bin = cv2pil(binarizer.run(image=pil2cv(region_image)))
region_image_bin = cv2pil(self.binarizer.binarize_image(image=pil2cv(region_image)))
region_image_bin_path = self.workspace.save_image_file(
region_image_bin,
"%s_%s.IMG-BIN" % (file_id, region.id),
Expand All @@ -139,7 +140,7 @@ def process(self):
LOG.warning("Page '%s' contains no text lines", page_id)
for region_id, line in region_line_tuples:
line_image, line_xywh = self.workspace.image_from_segment(line, page_image, page_xywh, feature_filter='binarized')
line_image_bin = cv2pil(binarizer.run(image=pil2cv(line_image)))
line_image_bin = cv2pil(self.binarizer.binarize_image(image=pil2cv(line_image)))
line_image_bin_path = self.workspace.save_image_file(
line_image_bin,
"%s_%s_%s.IMG-BIN" % (file_id, region_id, line.id),
Expand Down
Loading