Merge pull request #2 from xnuinside/v0.3.0_release

V0.3.0 release

Author: xnuinside

CHANGELOG.txt

@@ -1,3 +1,8 @@
+**v0.3.0**
+1. Added cli - `pmp` command with args -d, --dump  
+2. Added support for simple Django ORM models
+3. Added base support for pure Python Classes
+
 **v0.2.0**
 1. Added support for Dataclasses
 2. Added parse_from_file method

README.md

@@ -4,7 +4,19 @@
 
 
 It's as second Parser that done by me, first is a https://github.com/xnuinside/simple-ddl-parser for SQL DDL with different dialects.
-Py-Models-Parser supports now ORM Sqlalchemy, Gino, Tortoise; Pydantic, Python Enum models, Dataclasses & in nearest feature I plan to add pure pyton classes. And next will be added other ORMs models.
+
+Py-Models-Parser can parse & extract information from models:
+
+- Sqlalchemy ORM,
+- Gino ORM,
+- Tortoise ORM,
+- Django ORM Model,
+- Pydantic,
+- Python Enum,
+- Python Dataclasses
+- pure Python Classes
+
+Number of supported models will be increased, check 'TODO' section.
 
 Py-Models-Parser written with PEG parser and it's python implementation - parsimonious. It's pretty new and I did not cover all possible test cases, so if you will have an issue  - please just open an issue in this case with example, I will fix it as soon as possible.
 
@@ -51,7 +63,7 @@ NOTE: it's is a text parser, so it don't import or load your code, parser work w
 
 Library detect automaticaly that type of models you tries to parse. You can check a lot of examples in test/ folder on the GitHub
 
-You can parse models from python string:
+1. You can parse models from python string:
 
 ```python
 
@@ -75,7 +87,7 @@ result = parse(models_str)
 
 ```
 
-or just provide the path to file:
+2. Parse models from file:
 
 ```python
 
@@ -87,7 +99,31 @@ or just provide the path to file:
     result = parse_from_file(file_path)
 ```
 
-It will produce the result:
+3. Parse models from file with command line
+
+```bash
+
+    pmp path_to_models.py 
+
+    # for example: pmp tests/data/dataclass_defaults.py
+
+```
+
+Output from cli can be dumped in 'output_models.json' file - use flag '-d' '--dump' if you want to change target file name, provide it after argument like '-d target_file.json'
+
+```bash
+
+    # example how to dump output from cli
+
+    pmp path_to_models.py -d target_file.json
+
+```
+
+### Output example
+
+You can find a lot of output examples in tests - https://github.com/xnuinside/py-models-parser/tree/main/tests
+
+For model from point 1 (above) library will produce the result:
 
 ```python
 
@@ -132,16 +168,19 @@ It will produce the result:
     ]
 ```
 
-
 ## TODO: in next Release
 
-1. Parse from file method
-2. Add cli
-3. Add more tests for supported models (and fix existed not covered cases): Pydantic, Enums, Dataclasses, SQLAlchemy Models, GinoORM models, TortoiseORM models
-4. Add support for pure Python classes
-5. Add support for pure SQLAlchemy Core Tables
+1. Add more tests for supported models (and fix existed not covered cases): Django ORM, Pydantic, Enums, Dataclasses, SQLAlchemy Models, GinoORM models, TortoiseORM models
+2. Add support for SQLAlchemy Core Tables
+3. Add support for Pony ORM models
+4. Add support for Piccolo ORM models
 
 ## Changelog
+**v0.3.0**
+1. Added cli - `pmp` command with args -d, --dump  
+2. Added support for simple Django ORM models
+3. Added base support for pure Python Classes
+
 **v0.2.0**
 1. Added support for Dataclasses
 2. Added parse_from_file method

docs/README.rst

@@ -21,7 +21,20 @@ Py-Models-Parser
 
 
 It's as second Parser that done by me, first is a https://github.com/xnuinside/simple-ddl-parser for SQL DDL with different dialects.
-Py-Models-Parser supports now ORM Sqlalchemy, Gino, Tortoise; Pydantic, Python Enum models, Dataclasses & in nearest feature I plan to add pure pyton classes. And next will be added other ORMs models.
+
+Py-Models-Parser can parse & extract information from models:
+
+
+* Sqlalchemy ORM,
+* Gino ORM,
+* Tortoise ORM,
+* Django ORM Model,
+* Pydantic,
+* Python Enum,
+* Python Dataclasses
+* pure Python Classes
+
+Number of supported models will be increased, check 'TODO' section.
 
 Py-Models-Parser written with PEG parser and it's python implementation - parsimonious. It's pretty new and I did not cover all possible test cases, so if you will have an issue  - please just open an issue in this case with example, I will fix it as soon as possible.
 
@@ -69,7 +82,8 @@ How to use
 
 Library detect automaticaly that type of models you tries to parse. You can check a lot of examples in test/ folder on the GitHub
 
-You can parse models from python string:
+
+#. You can parse models from python string:
 
 .. code-block:: python
 
@@ -92,7 +106,8 @@ You can parse models from python string:
        """
    result = parse(models_str)
 
-or just provide the path to file:
+
+#. Parse models from file:
 
 .. code-block:: python
 
@@ -104,7 +119,31 @@ or just provide the path to file:
        # for example: tests/data/dataclass_defaults.py
        result = parse_from_file(file_path)
 
-It will produce the result:
+
+#. Parse models from file with command line
+
+.. code-block:: bash
+
+
+       pmp path_to_models.py 
+
+       # for example: pmp tests/data/dataclass_defaults.py
+
+Output from cli can be dumped in 'output_models.json' file - use flag '-d' '--dump' if you want to change target file name, provide it after argument like '-d target_file.json'
+
+.. code-block:: bash
+
+
+       # example how to dump output from cli
+
+       pmp path_to_models.py -d target_file.json
+
+Output example
+^^^^^^^^^^^^^^
+
+You can find a lot of output examples in tests - https://github.com/xnuinside/py-models-parser/tree/main/tests
+
+For model from point 1 (above) library will produce the result:
 
 .. code-block:: python
 
@@ -153,15 +192,21 @@ TODO: in next Release
 ---------------------
 
 
-#. Parse from file method
-#. Add cli
-#. Add more tests for supported models (and fix existed not covered cases): Pydantic, Enums, Dataclasses, SQLAlchemy Models, GinoORM models, TortoiseORM models
-#. Add support for pure Python classes
-#. Add support for pure SQLAlchemy Core Tables
+#. Add more tests for supported models (and fix existed not covered cases): Django ORM, Pydantic, Enums, Dataclasses, SQLAlchemy Models, GinoORM models, TortoiseORM models
+#. Add support for SQLAlchemy Core Tables
+#. Add support for Pony ORM models
+#. Add support for Piccolo ORM models
 
 Changelog
 ---------
 
+**v0.3.0**
+
+
+#. Added cli - ``pmp`` command with args -d, --dump  
+#. Added support for simple Django ORM models
+#. Added base support for pure Python Classes
+
 **v0.2.0**
 
 

py_models_parser/__init__.py

@@ -1,3 +1,3 @@
-from py_models_parser.core import parse, parse_from_file
+from py_models_parser.core import dump_result, parse, parse_from_file
 
-__all__ = ["parse", "parse_from_file"]
+__all__ = ["parse", "parse_from_file", "dump_result"]

py_models_parser/cli.py

@@ -0,0 +1,68 @@
+import argparse
+import os
+import pprint
+import sys
+
+from py_models_parser import dump_result, parse_from_file
+
+
+def version(**kwargs):
+    return "0.2.0"
+
+
+def cli():
+    pmp_cli = argparse.ArgumentParser(description="Python Models Parser")
+
+    pmp_cli.add_argument(
+        "file_path", type=str, help="The path to .py file with models to parse"
+    )
+
+    pmp_cli.add_argument(
+        "-d",
+        "--dump",
+        nargs="?",
+        type=str,
+        const="output_models.json",
+        help="Target path to save parse results in .json files",
+    )
+
+    return pmp_cli
+
+
+def run_for_file(args):
+    print(f"Start parsing file {args.file_path} \n")
+    result = parse_from_file(args.file_path)
+    if args.dump:
+        dump_result(result, args.dump)
+    print(f"File with result was saved to >> {args.dump} file")
+    if not args.dump:
+        pprint.pprint(result)
+
+
+def correct_extension(file_name: str) -> bool:
+    ext = ["ddl", "sql", "hql", ""]
+    split_name = file_name.split(".")
+    if len(split_name) >= 2:
+        ext_file = split_name[1]
+        if ext_file in ext:
+            return True
+    return False
+
+
+def main():
+    sdb_cli = cli()
+    args = sdb_cli.parse_args()
+    if not os.path.exists(args.file_path):
+        print("The file path specified does not exist")
+        sys.exit()
+    if os.path.isfile(args.file_path):
+        run_for_file(args)
+    else:
+        files = [
+            os.path.join(args.ddl_file_path, file_name)
+            for file_name in os.listdir(args.ddl_file_path)
+            if correct_extension(file_name)
+        ]
+        for file_path in files:
+            args.ddl_file_path = file_path
+            run_for_file(args)

py_models_parser/core.py

@@ -1,3 +1,4 @@
+import json
 import os
 from typing import Dict, List
 
@@ -76,3 +77,8 @@ def parse_from_file(file_path: str) -> List[Dict]:
     with open(file_path, "r") as f:
         models = f.read()
         return parse(models)
+
+
+def dump_result(output: str, file_path: str) -> None:
+    with open(file_path, "w+") as f:
+        json.dump(output, f, indent=1)

py_models_parser/grammar.py

@@ -2,21 +2,24 @@
 
 grammar = Grammar(
     r"""
-    expr = (class / call_result / attr_def / emptyline)*
-    class = class_def attr_def* ws?
+    expr = (class / call_result / attr_def / emptyline / funct_def)*
+    class = class_def attr_def* funct_def* ws?
     class_def   = intend? class_name args? ":"* ws?
-    attr_def  = intend? id ("=" right_part)* ws?
-    right_part = args / call_result / id / string / text
+    attr_def  = intend? id type? ("=" (right_part))* ws?
+    right_part =  (id args_in_brackets) / string / args  / call_result / args_in_brackets / id / text
+    type = ":" ( (id args_in_brackets) / id)
     string = one_quote_str / double_quotes_str
-    one_quote_str = ~"\'[^\']+\'"
-    double_quotes_str = ~'"[^\"]+"'
+    one_quote_str = ~"\'[^\']+\'"i
+    double_quotes_str = ~'"[^\"]+"'i
+    funct_def = intend? "def" id args? ":"* ws?
+    args_in_brackets = "[" ((id/string)* ","* )* "]"
     args     = "(" (( call_result / args / attr_def / id )* ","* )* ")"
     call_result = id args ws?
     class_name  = "class" id
-    id          = (((dot_id / text)+ ","*) *  / dot_id / text) ws?
+    id          = (((dot_id / text)+ ) *  / dot_id / text) ws?
     dot_id      = (text".")*text
-    intend      = "    " / "\t"
-    text        = !class ~"['\_A-Z 0-9\{\}\[\]_\"\-\/\$:<%>\w]*"i
+    intend      = "    " / "\t" / "\n"
+    text        = !class ~"['\_A-Z 0-9\{\}_\"\-\/\$<%>\+\-\w]*"i
     ws          = ~"\s*"
     emptyline   = ws+
 """