Skip to content

Update to the new MkDoxy config #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 19, 2025
Merged

Conversation

JakubAndrysek
Copy link
Owner

@JakubAndrysek JakubAndrysek commented Feb 18, 2025

Update to the new MkDoxy config

Summary by Sourcery

This pull request updates the MkDoxy plugin to use a new configuration system, improves Doxygen configuration, implements change detection, and adds support for generating diagrams. It also refactors the plugin to use a DoxygenGenerator class for managing Doxygen execution and configuration.

New Features:

  • Introduces a new configuration system using MkDoxyConfig and MkDoxyConfigProject for global and project-specific settings, respectively.
  • Adds support for generating diagrams using Doxygen's DOT functionality, with configurable formats and types (DOT or UML).

Enhancements:

  • Refactors the plugin to use a DoxygenGenerator class for managing Doxygen execution and configuration.
  • Improves Doxygen configuration by allowing a combination of default settings, custom configuration files, and inline configuration options.
  • Implements change detection using SHA1 hashing to avoid unnecessary Doxygen runs.
  • Updates template loading to use JINJA_EXTENSIONS constant for supported file endings.

Build:

  • Removes devdeps.txt file.

Chores:

  • Removes the DoxygenRun class and replaces it with DoxygenGenerator.
  • Removes the need for temporary directories by allowing the user to specify a custom API folder.
  • Removes the need for a separate config schema, and uses the new MkDoxyConfig and MkDoxyConfigProject instead.

Copy link

sourcery-ai bot commented Feb 18, 2025

Reviewer's Guide by Sourcery

This pull request introduces a new configuration system, refactors Doxygen execution and XML parsing, and updates template handling and file processing. The changes improve the plugin's flexibility, maintainability, and error handling.

Updated class diagram for MkDoxy plugin

classDiagram
    class MkDoxyConfig {
        +projects: DictOfItems
        +full_doc: bool
        +debug: bool
        +ignore_errors: bool
        +custom_api_folder: Optional[str]
        +doxygen_bin_path: Path
        +generate_diagrams: bool
        +generate_diagrams_format: Choice
        +generate_diagrams_type: Choice
    }
    class MkDoxyConfigProject {
        +src_dirs: str
        +full_doc: bool
        +debug: bool
        +ignore_errors: bool
        +doxy_config_dict: dict
        +doxy_config_file: Optional[Path]
        +custom_template_dir: Optional[Path]
    }

    MkDoxyConfig *-- MkDoxyConfigProject : projects
Loading

Class diagram for DoxygenGenerator

classDiagram
    class DoxygenGenerator {
        -doxy_config: MkDoxyConfig
        -project_config: MkDoxyConfigProject
        -temp_doxy_folder: Path
        +__init__(doxy_config: MkDoxyConfig, project_config: MkDoxyConfigProject, temp_doxy_folder: Path)
        +set_doxy_config(doxy_cfg_new: dict) : dict
        +diagram_update_config(doxy_config)
        +is_doxygen_valid_path(doxygen_bin_path: Path) : bool
        +dox_dict2str(dox_dict: dict) : str
        +str2dox_dict(dox_str: str) : dict
        +hash_write(file_name: Path, hash_key: str)
        +hash_read(file_name: Path) : str
        +has_changes() : bool
        +run() : None
        +get_output_xml_folder() : Path
        +get_output_html_folder() : Path
    }

    DoxygenGenerator -- MkDoxyConfig : doxy_config
    DoxygenGenerator -- MkDoxyConfigProject : project_config
Loading

File-Level Changes

Change Details Files
Introduced a new configuration system using MkDoxyConfig and MkDoxyConfigProject for global and project-specific settings, respectively.
  • Replaced the old config scheme with MkDoxyConfig and MkDoxyConfigProject.
  • Added validation for the new configuration options.
  • Deprecated the old configuration options.
  • Added support for custom Doxygen configuration files and dictionaries.
  • Added support for custom Jinja2 template directories.
mkdoxy/plugin.py
mkdocs.yml
mkdoxy/doxy_config.py
Refactored Doxygen execution and XML parsing into a DoxygenGenerator class.
  • Created a DoxygenGenerator class to handle Doxygen execution and configuration.
  • Implemented change detection using SHA1 hashing to avoid unnecessary Doxygen runs.
  • Improved error handling for Doxygen execution and configuration.
  • Added support for generating diagrams using Doxygen and Graphviz.
mkdoxy/plugin.py
mkdoxy/doxygen_generator.py
mkdoxy/doxy_config.py
mkdocs.yml
Updated template handling and file processing.
  • Updated the way templates are loaded and used.
  • Ensured that generated files are correctly appended to the MkDocs files collection.
  • Fixed an issue where files were being added multiple times to the MkDocs files collection.
  • Updated the way custom templates are loaded and overwritten.
mkdoxy/plugin.py
mkdoxy/generatorBase.py
mkdoxy/constants.py
Removed the DoxygenRun class.
  • Removed the DoxygenRun class.
  • Moved the functionality of the DoxygenRun class to the DoxygenGenerator class.
mkdoxy/plugin.py
mkdoxy/doxyrun.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @JakubAndrysek - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider adding a migration guide or deprecation warnings for users upgrading to this new configuration.
  • It would be helpful to include some documentation or examples of the new configuration format.
Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

for line in dox_str.split("\n"):
if line.strip() == "":
continue
key, value = line.split(" = ")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Use a safer split for key-value parsing.

To prevent potential issues when a value contains the delimiter, consider specifying a maximum split (e.g. line.split(" = ", 1)) to ensure only the first occurrence is used for splitting.

Suggested change
key, value = line.split(" = ")
key, value = line.split(" = ", 1)

Comment on lines +224 to +227
sha1 = hashlib.sha1()
sources = self.project_config.src_dirs.split(" ")
# Code from https://stackoverflow.com/a/22058673/15411117
BUF_SIZE = 65536 # let's read stuff in 64kb chunks!
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Clarify delimiter used for src_dirs splitting.

Currently, the code splits src_dirs on spaces. It could be beneficial to either document this delimiter in the configuration or consider supporting a more robust format (e.g. comma-separated values) for cases where directory paths might include spaces.

Suggested change
sha1 = hashlib.sha1()
sources = self.project_config.src_dirs.split(" ")
# Code from https://stackoverflow.com/a/22058673/15411117
BUF_SIZE = 65536 # let's read stuff in 64kb chunks!
sha1 = hashlib.sha1()
# Split src_dirs as comma-separated values; directory names containing spaces are allowed.
sources = [s.strip() for s in self.project_config.src_dirs.split(",") if s.strip()]
# Code from https://stackoverflow.com/a/22058673/15411117
BUF_SIZE = 65536 # let's read stuff in 64kb chunks!

f"Look at https://mkdoxy.kubaandrysek.cz/usage/advanced/#configure-custom-doxygen-binary."
)

def set_doxy_config(self, doxy_cfg_new: dict) -> dict:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity): Consider refactoring large methods like set_doxy_config and has_changes into smaller, more focused helper functions.

Consider refactoring some of the larger methods into focused helper functions to reduce nesting and isolate concerns. For example:

  1. Extract Custom Config Loading
    Move the custom config file logic from set_doxy_config into its own method. This reduces nesting and clarifies intent. For instance:

    def _load_custom_config(self) -> dict:
        if self.project_config.doxy_config_file is None:
            return {}
        if not self.project_config.doxy_config_file.is_file():
            raise DoxygenCustomConfigNotFound(
                f"Custom Doxygen config file not found: {self.project_config.doxy_config_file}\n"
                f"Make sure the path is correct. Loaded path: '{self.project_config.doxy_config_file}'"
            )
        with open(self.project_config.doxy_config_file, "r") as file:
            return self.str2dox_dict(file.read())

    Then update set_doxy_config:

    def set_doxy_config(self, doxy_cfg_new: dict) -> dict:
        if self.project_config.doxy_config_file:
            doxy_config = self._load_custom_config()
        else:
            doxy_config = {
                "DOXYFILE_ENCODING": "UTF-8",
                "GENERATE_XML": "YES",
                "RECURSIVE": "YES",
                "EXAMPLE_PATH": "examples",
                "SHOW_NAMESPACES": "YES",
                "GENERATE_HTML": "NO",
                "GENERATE_LATEX": "NO",
            }
        doxy_config.update(doxy_cfg_new)
        if self.doxy_config.generate_diagrams:
            self.diagram_update_config(doxy_config)
        doxy_config["INPUT"] = self.project_config.src_dirs
        doxy_config["OUTPUT_DIRECTORY"] = str(self.temp_doxy_folder)
        return doxy_config
  2. Extract File Hash Calculation
    Factor out the hashing logic of has_changes for clarity:

    def _compute_sources_hash(self) -> str:
        sha1 = hashlib.sha1()
        BUF_SIZE = 65536
        for source in self.project_config.src_dirs.split(" "):
            for path in Path(source).rglob("*.*"):
                if path.is_file():
                    with open(path, "rb") as file:
                        for data in iter(lambda: file.read(BUF_SIZE), b""):
                            sha1.update(data)
        return sha1.hexdigest()

    Then modify has_changes as follows:

    def has_changes(self) -> bool:
        hash_new = self._compute_sources_hash()
        hash_file_path = self.temp_doxy_folder.joinpath("mkdoxy_hash.txt")
        if hash_file_path.is_file():
            hash_old = self.hash_read(hash_file_path)
            if hash_new == hash_old:
                return False  # No changes in the source files
        self.hash_write(hash_file_path, hash_new)
        return True

These small, focused methods help reduce procedural nesting and make each piece of functionality easier to test and maintain without changing the existing behavior.

@JakubAndrysek JakubAndrysek merged commit c012a5c into core-update Feb 19, 2025
26 checks passed
@JakubAndrysek JakubAndrysek deleted the core-update-config branch February 19, 2025 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant