Skip to content

Commit 6ad0b83

Browse files
authored
Release v2.3.0, add GitHub Actions and SonarCloud
* add normalize_images method as an abstract method in image_processors.py * fix: change OpenCVVideo method name get_next_video_frames -> get_next_frames * move out normalizing images from evalutor to extractors * rename get_extractor() -> create_extractor() in ExtractorFactory * add license to docstrings * move ServiceShutdownSignal inside DockerManager * make ServiceInitializer attributes protected * add architecture section to README * add created status to manager and tests * add cpu-only flag * add new quick_demo.bat files * add new flag to readme * change requirements in readme * fix paths in docker-compose.yaml and change method 2 in readme * add 'created at' badge * change _get_image_evaluator() to returns ImageEvaluator insted of specific evaluator * fix DIP in extractors * add pylint to dependendcies * move extractor service tests to root dir and fix imports * move service_manager tests to root folder tests and fix paths * fix extractors unit tests after fixing DIP in extractors * move dependency injection to extractors manager * add dependencies module * change README files tests sections after moving tests to root folder * use Depends() from FastAPI for better dependencies handling and refactor code and tests * add tests for dependencies.py * small changes in docstings, small refactor code changes * change logging level to INFO * change cpu_only to True in integration tests in service_manager * add --cpu flag to service_manager e2e tests * add .gitkeep to test_files * fix test_video * fix paths in test_add_prefix * Add local server to run_tests.yml * add skipif in service_manager e2e tests * remove additional server in github actions * change http to https * add assert before isinstance() * change 'is not None' to just 'assert extractor' * fix logging check in test_list_input_directory_files * Fix assertion for _dropout_rate to float with tolerance * Add comment before pass in test_get_video_capture_failure for clarity * remove magic values in test_get_next_video_frames * remove overwriting client variable in image fixture * fix removing test folders after tests * add sleep 300 command as a constant for handling DRY * fix test_container_status * save log_lines as constants for handling DRY * remove unused variable pop * Remove the unused local variable 'container' * add test video
1 parent 54fe463 commit 6ad0b83

File tree

70 files changed

+712
-304
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+712
-304
lines changed

.github/README.pl.md

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
</div>
44
<div id="badges">
55
<p align="center">
6+
<img alt="Github Created At" src="https://img.shields.io/github/created-at/BKDDFS/PerfectFrameAI">
67
<img alt="GitHub Downloads (all assets, all releases)" src="https://img.shields.io/github/downloads/BKDDFS/PerfectFrameAI/total?style=flat&color=blue">
78
<img alt="GitHub License" src="https://img.shields.io/github/license/BKDDFS/PerfectFrameAI">
89
<img alt="GitHub Release" src="https://img.shields.io/github/v/release/BKDDFS/PerfectFrameAI">
@@ -182,6 +183,24 @@
182183
<td>bool</td>
183184
<td>False</td>
184185
</tr>
186+
<tr>
187+
<td>--all_frames</td>
188+
<td></td>
189+
<td>
190+
Do pomijania oceniania klatek.
191+
</td>
192+
<td>bool</td>
193+
<td>False</td>
194+
</tr>
195+
<tr>
196+
<td>--cpu</td>
197+
<td></td>
198+
<td>
199+
Wyłącza korzystanie z GPU. Musisz tego użyć jeśli nie masz GPU.
200+
</td>
201+
<td>bool</td>
202+
<td>False</td>
203+
</tr>
185204
</tbody>
186205
</table>
187206
<p><strong>Przykład dla Best Frames Extraction:</strong></p>
@@ -434,11 +453,6 @@
434453
Testy możesz uruchomić instalując zależności z <code>pyproject.toml</code>
435454
i wpisując w terminal w lokalizacj projektu - <code>pytest</code>.
436455
</p>
437-
<blockquote>
438-
Proszę zwrócić uwagę, że w projekcie są dwa foldery <code>tests/</code>.
439-
<code>extractor_service</code> i <code>service_initializer</code> mają testy osobno.
440-
W pliku common.py znajdują się pliki wpółdzielone przez testy i potrzebne do ich działania.
441-
</blockquote>
442456
<details id="unit">
443457
<summary>jednostkowe</summary>
444458
<p>

.gitignore

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@ input_directory/*
99
output_directory/*
1010
!input_directory/.gitkeep
1111
!output_directory/.gitkeep
12-
test_video.mp4
1312
nima.h5
14-
common/test_files/best_frames/*
15-
common/test_files/top_images/*
13+
tests/test_files/best_frames/*
14+
tests/test_files/top_images/*
15+
!tests/test_files/best_frames/.gitkeep
16+
!tests/test_files/top_images/.gitkeep

README.md

Lines changed: 38 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
</div>
44
<div id="badges">
55
<p align="center">
6+
<img alt="Github Created At" src="https://img.shields.io/github/created-at/BKDDFS/PerfectFrameAI">
67
<img alt="GitHub last commit" src="https://img.shields.io/github/last-commit/BKDDFS/PerfectFrameAI">
78
<img alt="GitHub License" src="https://img.shields.io/github/license/BKDDFS/PerfectFrameAI">
89
<img alt="GitHub Tag" src="https://img.shields.io/github/v/tag/BKDDFS/PerfectFrameAI">
@@ -106,17 +107,19 @@
106107
<h3>System Requirements:</h3>
107108
<ul>
108109
<li>Docker</li>
109-
<li>Python 3.10 or higher (method 1 only)</li>
110-
<li>Nvidia GPU (recommended)</li>
111-
<li>10 GB free disk space</li>
112-
</ul>
110+
<li>Python 3.7+ (method 1 only)</li>
111+
<li>8GB+ RAM</li>
112+
<li>10GB+ free disk space</li>
113+
</ul>
114+
<p>Lowest tested specs - i5-4300U, 8GB RAM (ThinkPad T440) - 4k video, default 100img/batch.</p>
115+
<p>Remember you can always decrease images batch size in schemas.py if you out of RAM.</p>
113116
</blockquote>
114117
<details>
115118
<summary>Install Docker:</summary>
116119
Docker Desktop: <a href="https://www.docker.com/products/docker-desktop/">https://www.docker.com/products/docker-desktop/</a>
117120
</details>
118121
<details>
119-
<summary>Install Python v3.10+:</summary>
122+
<summary>Install Python v3.7+:</summary>
120123
MS Store: <a href="https://apps.microsoft.com/detail/9ncvdn91xzqp?hl=en-US&gl=US">https://apps.microsoft.com/detail/9ncvdn91xzqp?hl=en-US&gl=US</a><br>
121124
Python.org: <a href="https://www.python.org/downloads/">https://www.python.org/downloads/</a>
122125
</details>
@@ -140,9 +143,17 @@
140143
<blockquote>
141144
<p>
142145
<strong>Hint for Windows users:</strong><br>
143-
As a Windows user you can use <code>quick_demo.bat</code> file.
144-
It will run <code>best_frames_extractor</code> with the default values. Just double click on it.
145-
You can modify default values in config.py to adjust the application to your needs.
146+
As a Windows user, you can use:<br>
147+
<code>quick_demo_gpu.bat</code> or <code>quick_demo_cpu.bat</code>
148+
if you don't have an Nvidia GPU.<br>
149+
It will run <code>best_frames_extractor</code> with the default values.
150+
Just double-click on it.
151+
You can modify the default values in config.py to adjust the application to your needs.<br>
152+
<strong>Warning!</strong><br>
153+
Please note that when running the .bat file,
154+
Windows Defender may flag it as dangerous.
155+
This happens because obtaining a code-signing certificate
156+
to prevent this warning requires a paid certificate...
146157
</p>
147158
</blockquote>
148159
<p>Run <code>start.py</code> from the terminal.</p>
@@ -191,6 +202,24 @@
191202
<td>bool</td>
192203
<td>False</td>
193204
</tr>
205+
<tr>
206+
<td>--all_frames</td>
207+
<td></td>
208+
<td>
209+
For skipping frames evaluation part.
210+
</td>
211+
<td>bool</td>
212+
<td>False</td>
213+
</tr>
214+
<tr>
215+
<td>--cpu</td>
216+
<td></td>
217+
<td>
218+
Uses only CPU for processing. If you, don't have GPU you must use it.
219+
</td>
220+
<td>bool</td>
221+
<td>False</td>
222+
</tr>
194223
</tbody>
195224
</table>
196225
<p><strong>Example (Best Frames Extraction):</strong></p>
@@ -203,6 +232,7 @@
203232
<blockquote><p><i>Does not require Python. Run using Docker Compose.</i></p></blockquote>
204233
</summary>
205234
<p>Docker Compose Docs: <a href="https://docs.docker.com/compose/">https://docs.docker.com/compose/</a></p>
235+
<p>Remember to delete GPU part in docker-compose.yaml if you don't have GPU!</p>
206236
<ol>
207237
<li>Run the service: <br><code>docker-compose up --build -d</code></li>
208238
<li>Send a request to the chosen endpoint.
@@ -415,11 +445,6 @@
415445
You can run the tests by installing the dependencies from <code>pyproject.toml</code>
416446
and typing in the terminal in the project location - <code>pytest</code>.
417447
</p>
418-
<blockquote>
419-
Please note that there are two <code>tests/</code> folders in the project.
420-
<code>extractor_service</code> and <code>service_initializer</code> have separate tests.
421-
The common.py file contains shared files for the tests and necessary for their operation.
422-
</blockquote>
423448
<details id="unit">
424449
<summary>unit</summary>
425450
<p>

docker-compose.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ services:
66
ports:
77
- "8100:8100"
88
volumes:
9-
- "B:/frames_evaluators/input_directory:/app/input_directory"
10-
- "B:/frames_evaluators/output_directory:/app/output_directory"
9+
- "./input_directory:/app/input_directory"
10+
- "./output_directory:/app/output_directory"
1111
environment:
1212
- NVIDIA_VISIBLE_DEVICES=all
1313
- NVIDIA_DRIVER_CAPABILITIES=compute,video,utility

extractor_service/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ RUN pip install --no-cache-dir -r requirements.txt
2929
ENV NVIDIA_VISIBLE_DEVICES all
3030
ENV NVIDIA_DRIVER_CAPABILITIES compute,video,utility
3131
ENV TF_CPP_MIN_LOG_LEVEL 3
32+
ENV DOCKER_ENV=1
3233

3334
COPY . .
3435

File renamed without changes.
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
"""
2+
This module provides dependency management for extractors using FastAPI's dependency injection.
3+
LICENSE
4+
=======
5+
Copyright (C) 2024 Bartłomiej Flis
6+
7+
This program is free software: you can redistribute it and/or modify
8+
it under the terms of the GNU General Public License as published by
9+
the Free Software Foundation, either version 3 of the License, or
10+
(at your option) any later version.
11+
12+
This program is distributed in the hope that it will be useful,
13+
but WITHOUT ANY WARRANTY; without even the implied warranty of
14+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15+
GNU General Public License for more details.
16+
17+
You should have received a copy of the GNU General Public License
18+
along with this program. If not, see <https://www.gnu.org/licenses/>.
19+
"""
20+
from dataclasses import dataclass
21+
from typing import Type
22+
23+
from fastapi import Depends
24+
25+
from .image_evaluators import InceptionResNetNIMA
26+
from .image_processors import OpenCVImage
27+
from .video_processors import OpenCVVideo
28+
29+
30+
@dataclass
31+
class ExtractorDependencies:
32+
"""
33+
Data class to hold dependencies for the extractor.
34+
35+
Attributes:
36+
image_processor (Type[OpenCVImage]): Processor for image processing.
37+
video_processor (Type[OpenCVVideo]): Processor for video processing.
38+
evaluator (Type[InceptionResNetNIMA]): Evaluator for image quality.
39+
"""
40+
image_processor: Type[OpenCVImage]
41+
video_processor: Type[OpenCVVideo]
42+
evaluator: Type[InceptionResNetNIMA]
43+
44+
45+
def get_image_processor() -> Type[OpenCVImage]:
46+
"""
47+
Provides the image processor dependency.
48+
49+
Returns:
50+
Type[OpenCVImage]: The image processor class.
51+
"""
52+
return OpenCVImage
53+
54+
55+
def get_video_processor() -> Type[OpenCVVideo]:
56+
"""
57+
Provides the video processor dependency.
58+
59+
Returns:
60+
Type[OpenCVVideo]: The video processor class.
61+
"""
62+
return OpenCVVideo
63+
64+
65+
def get_evaluator() -> Type[InceptionResNetNIMA]:
66+
"""
67+
Provides the image evaluator dependency.
68+
69+
Returns:
70+
Type[InceptionResNetNIMA]: The image evaluator class.
71+
"""
72+
return InceptionResNetNIMA
73+
74+
75+
def get_extractor_dependencies(
76+
image_processor=Depends(get_image_processor),
77+
video_processor=Depends(get_video_processor),
78+
evaluator=Depends(get_evaluator)
79+
) -> ExtractorDependencies:
80+
"""
81+
Provides the dependencies required for the extractor.
82+
83+
Args:
84+
image_processor (Type[OpenCVImage], optional): Dependency injection for image processor.
85+
video_processor (Type[OpenCVVideo], optional): Dependency injection for video processor.
86+
evaluator (Type[InceptionResNetNIMA], optional): Dependency injection for image evaluator.
87+
88+
Returns:
89+
ExtractorDependencies: All necessary dependencies for the extractor.
90+
"""
91+
return ExtractorDependencies(
92+
image_processor=image_processor,
93+
video_processor=video_processor,
94+
evaluator=evaluator
95+
)

extractor_service/app/extractor_manager.py

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@
1919
along with this program. If not, see <https://www.gnu.org/licenses/>.
2020
"""
2121
import logging
22-
from typing import Type
2322

2423
from fastapi import HTTPException, BackgroundTasks
2524

25+
from .dependencies import ExtractorDependencies
2626
from .extractors import Extractor, ExtractorFactory
2727
from .schemas import ExtractorConfig
2828

@@ -35,7 +35,6 @@ class ExtractorManager:
3535
maintaining system stability.
3636
"""
3737
_active_extractor = None
38-
_config = None
3938

4039
@classmethod
4140
def get_active_extractor(cls) -> str:
@@ -48,30 +47,28 @@ def get_active_extractor(cls) -> str:
4847
return cls._active_extractor
4948

5049
@classmethod
51-
def start_extractor(cls, background_tasks: BackgroundTasks, config: ExtractorConfig,
52-
extractor_name: str) -> str:
50+
def start_extractor(cls, extractor_name: str, background_tasks: BackgroundTasks,
51+
config: ExtractorConfig, dependencies: ExtractorDependencies) -> str:
5352
"""
5453
Initializes the extractor class and runs the extraction process in the background.
5554
5655
Args:
57-
config (ExtractorConfig): A Pydantic model with configuration
58-
parameters for the extractor.
59-
background_tasks: A FastAPI tool for running tasks in background,
60-
which allows non-blocking operation of long-running tasks.
6156
extractor_name (str): The name of the extractor that will be used.
57+
background_tasks (BackgroundTasks): A FastAPI tool for running tasks in background.
58+
config (ExtractorConfig): A Pydantic model with extractor configuration.
59+
dependencies(ExtractorDependencies): Dependencies that will be used in extractor.
6260
6361
Returns:
6462
str: Endpoint feedback message with started extractor name.
6563
"""
66-
cls._config = config
6764
cls._check_is_already_extracting()
68-
extractor_class = ExtractorFactory.create_extractor(extractor_name)
69-
background_tasks.add_task(cls.__run_extractor, extractor_class, extractor_name)
65+
extractor = ExtractorFactory.create_extractor(extractor_name, config, dependencies)
66+
background_tasks.add_task(cls.__run_extractor, extractor, extractor_name)
7067
message = f"'{extractor_name}' started."
7168
return message
7269

7370
@classmethod
74-
def __run_extractor(cls, extractor: Type[Extractor], extractor_name: str) -> None:
71+
def __run_extractor(cls, extractor: Extractor, extractor_name: str) -> None:
7572
"""
7673
Run extraction process and clean after it's done.
7774
@@ -81,10 +78,9 @@ def __run_extractor(cls, extractor: Type[Extractor], extractor_name: str) -> Non
8178
"""
8279
try:
8380
cls._active_extractor = extractor_name
84-
extractor(cls._config).process()
81+
extractor.process()
8582
finally:
8683
cls._active_extractor = None
87-
cls._config = None
8884

8985
@classmethod
9086
def _check_is_already_extracting(cls) -> None:

0 commit comments

Comments
 (0)