Update req

BenConstable9 · BenConstable9 · commit 63c4915d1931 · 2025-01-22T19:30:49.000Z
diff --git a/deploy_ai_search_indexes/README.md b/deploy_ai_search_indexes/README.md
@@ -4,22 +4,37 @@ The associated scripts in this portion of the repository contains pre-built scri
 
 ## Steps for Image Processing Index Deployment (For Image Processing)
 
-1. Update `.env` file with the associated values. Not all values are required dependent on whether you are using System / User Assigned Identities or a Key based authentication.
-2. Adjust `image_processing.py` with any changes to the index / indexer. The `get_skills()` method implements the skills pipeline. Make any adjustments here in the skills needed to enrich the data source.
-3. Run `deploy.py` with the following args:
+**Execute the following commands in the `deploy_ai_search_indexes` directory:**
 
+1. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
+2. Run `uv sync` within the `deploy_ai_search_indexes` directory to install dependencies.
+    - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
+    - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
+
+**Execute the following commands in the `deploy_ai_search_indexes/src/deploy_ai_search_indexes` directory:**
+
+3. Adjust `image_processing.py` with any changes to the index / indexer. The `get_skills()` method implements the skills pipeline. Make any adjustments here in the skills needed to enrich the data source.
+4. Run `deploy.py` with the following args:
     - `index_type image_processing`. This selects the `ImageProcessingAISearch` sub class.
-    - `enable_page_chunking True`. This determines whether page wise chunking is applied in ADI, or whether the inbuilt skill is used for TextSplit. This suits documents that are inheritely page-wise e.g. pptx files.
+    - `enable_page_wise_chunking True`. This determines whether page wise chunking is applied in ADI, or whether the inbuilt skill is used for TextSplit. This suits documents that are inheritely page-wise e.g. pptx files.
     - `rebuild`. Whether to delete and rebuild the index.
     - `suffix`. Optional parameter that will apply a suffix onto the deployed index and indexer. This is useful if you want deploy a test version, before overwriting the main version.
 
 ## Steps for Text2SQL Index Deployment (For Text2SQL)
 
 ### Schema Store Index
 
-1. Update `.env` file with the associated values. Not all values are required dependent on whether you are using System / User Assigned Identities or a Key based authentication.
-2. Adjust `text_2_sql_schema_store.py` with any changes to the index / indexer. The `get_skills()` method implements the skills pipeline. Make any adjustments here in the skills needed to enrich the data source.
-3. Run `deploy.py` with the following args:
+**Execute the following commands in the `deploy_ai_search_indexes` directory:**
+
+1. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
+2. Run `uv sync` within the `deploy_ai_search_indexes` directory to install dependencies.
+    - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
+    - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
+
+**Execute the following commands in the `deploy_ai_search_indexes/src/deploy_ai_search_indexes` directory:**
+
+3. Adjust `text_2_sql_schema_store.py` with any changes to the index / indexer. The `get_skills()` method implements the skills pipeline. Make any adjustments here in the skills needed to enrich the data source.
+4. Run `deploy.py` with the following args:
 
     - `index_type text_2_sql_schema_store`. This selects the `Text2SQLSchemaStoreAISearch` sub class.
     - `rebuild`. Whether to delete and rebuild the index.
@@ -28,19 +43,35 @@ The associated scripts in this portion of the repository contains pre-built scri
 
 ### Column Value Store Index
 
-1. Update `.env` file with the associated values. Not all values are required dependent on whether you are using System / User Assigned Identities or a Key based authentication.
-2. Adjust `text_2_sql_column_value_store.py` with any changes to the index / indexer.
-3. Run `deploy.py` with the following args:
+**Execute the following commands in the `deploy_ai_search_indexes` directory:**
+
+1. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
+2. Run `uv sync` within the `deploy_ai_search_indexes` directory to install dependencies.
+    - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
+    - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
+
+**Execute the following commands in the `deploy_ai_search_indexes/src/deploy_ai_search_indexes` directory:**
+
+3. Adjust `text_2_sql_column_value_store.py` with any changes to the index / indexer.
+4. Run `deploy.py` with the following args:
 
     - `index_type text_2_sql_column_value_store`. This selects the `Text2SQLColumnValueStoreAISearch` sub class.
     - `rebuild`. Whether to delete and rebuild the index.
     - `suffix`. Optional parameter that will apply a suffix onto the deployed index and indexer. This is useful if you want deploy a test version, before overwriting the main version.
 
 ### Query Cache Index
 
-1. Update `.env` file with the associated values. Not all values are required dependent on whether you are using System / User Assigned Identities or a Key based authentication.
-2. Adjust `text_2_sql_query_cache.py` with any changes to the index. **There is an optional provided indexer or skillset for this cache. You may instead want the application code will write directly to it. See the details in the Text2SQL README for different cache strategies.**
-3. Run `deploy.py` with the following args:
+**Execute the following commands in the `deploy_ai_search_indexes` directory:**
+
+1. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
+2. Run `uv sync` within the `deploy_ai_search_indexes` directory to install dependencies.
+    - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
+    - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
+
+**Execute the following commands in the `deploy_ai_search_indexes/src/deploy_ai_search_indexes` directory:**
+
+3. Adjust `text_2_sql_query_cache.py` with any changes to the index. **There is an optional provided indexer or skillset for this cache. You may instead want the application code will write directly to it. See the details in the Text2SQL README for different cache strategies.**
+4. Run `deploy.py` with the following args:
 
     - `index_type text_2_sql_query_cache`. This selects the `Text2SQLQueryCacheAISearch` sub class.
     - `rebuild`. Whether to delete and rebuild the index.
diff --git a/deploy_ai_search_indexes/src/deploy_ai_search_indexes/deploy.py b/deploy_ai_search_indexes/src/deploy_ai_search_indexes/deploy.py
@@ -21,7 +21,7 @@ def deploy_config(arguments: argparse.Namespace):
         index_config = ImageProcessingAISearch(
             suffix=suffix,
             rebuild=arguments.rebuild,
-            enable_page_by_chunking=arguments.enable_page_chunking,
+            enable_page_by_chunking=arguments.enable_page_wise_chunking,
         )
     elif arguments.index_type == "text_2_sql_schema_store":
         index_config = Text2SqlSchemaStoreAISearch(
@@ -65,7 +65,7 @@ def deploy_config(arguments: argparse.Namespace):
         help="Whether want to delete and rebuild the index",
     )
     parser.add_argument(
-        "--enable_page_chunking",
+        "--enable_page_wise_chunking",
         type=bool,
         required=False,
         help="Whether want to enable chunking by page in adi skill, if no value is passed considered False",
diff --git a/deploy_ai_search_indexes/src/deploy_ai_search_indexes/image_processing.py b/deploy_ai_search_indexes/src/deploy_ai_search_indexes/image_processing.py
@@ -243,8 +243,8 @@ def get_index_projections(self) -> SearchIndexerIndexProjection:
                     source="/document/page_wise_layout/*/chunk_sections",
                 ),
                 InputFieldMappingEntry(
-                    name="Figures",
-                    source_context="/document/page_wise_layout/*/chunk_figures/*",
+                    name="ChunkFigures",
+                    source="/document/page_wise_layout/*/chunk_figures/*",
                 ),
                 InputFieldMappingEntry(
                     name="DateLastModified", source="/document/DateLastModified"
@@ -269,8 +269,8 @@ def get_index_projections(self) -> SearchIndexerIndexProjection:
                     name="Sections", source="/document/chunk_mark_ups/*/chunk_sections"
                 ),
                 InputFieldMappingEntry(
-                    name="Figures",
-                    source_context="/document/chunk_mark_ups/*/chunk_figures/*",
+                    name="ChunkFigures",
+                    source="/document/chunk_mark_ups/*/chunk_figures/*",
                 ),
                 InputFieldMappingEntry(
                     name="DateLastModified", source="/document/DateLastModified"
diff --git a/image_processing/src/image_processing/requirements.txt b/image_processing/src/image_processing/requirements.txt
@@ -44,7 +44,7 @@ annotated-types==0.7.0 \
 anyio==4.8.0 \
     --hash=sha256:1d9fe889df5212298c0c0723fa20479d1b94883a2df44bd3897aa91083316f7a \
     --hash=sha256:b5011f270ab5eb0abf13385f851315585cc37ef330dd88e27ec3d34d651fd47a
-appnope==0.1.4 ; sys_platform == 'darwin' \
+appnope==0.1.4 ; platform_system == 'Darwin' \
     --hash=sha256:1de3860566df9caf38f01f86f65e0e13e379af54f9e4bee1e66b48f2efffd1ee \
     --hash=sha256:502575ee11cd7a28c0205f379b525beefebab9d161b7c964670864014ed7213c
 argon2-cffi==23.1.0 \
@@ -202,7 +202,7 @@ click==8.1.8 \
 cloudpathlib==0.20.0 \
     --hash=sha256:7af3bcefbf73392ae7f31c08b3660ec31607f8c01b7f6262d4d73469a845f641 \
     --hash=sha256:f6ef7ca409a510f7ba4639ba50ab3fc5b6dee82d6dff0d7f5715fd0c9ab35891
-colorama==0.4.6 ; sys_platform == 'win32' \
+colorama==0.4.6 ; sys_platform == 'win32' or platform_system == 'Windows' \
     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
 comm==0.2.2 \
@@ -277,9 +277,9 @@ en-core-web-md @ https://github.com/explosion/spacy-models/releases/download/en_
 et-xmlfile==2.0.0 \
     --hash=sha256:7a91720bc756843502c3b7504c77b8fe44217c85c537d85037f0f536151b2caa \
     --hash=sha256:dab3f4764309081ce75662649be815c4c9081e88f0837825f90fd28317d4da54
-executing==2.1.0 \
-    --hash=sha256:8d63781349375b5ebccc3142f4b30350c0cd9c79f921cde38be2be4637e98eaf \
-    --hash=sha256:8ea27ddd260da8150fa5a708269c4a10e76161e2496ec3e587da9e3c0fe4b9ab
+executing==2.2.0 \
+    --hash=sha256:11387150cad388d62750327a53d3339fad4888b39a6fe233c3afbb54ecffd3aa \
+    --hash=sha256:5d108c028108fe2551d1a7b2e8b713341e2cb4fc0aa7dcf966fa4327a5226755
 fastjsonschema==2.21.1 \
     --hash=sha256:794d4f0a58f848961ba16af7b9c85a3e88cd360df008c59aac6fc5ae9323b5d4 \
     --hash=sha256:c9e5b7e908310918cf494a434eeb31384dd84a98b57a30bcb1f535015b554667
@@ -616,9 +616,9 @@ numpy==1.26.4 \
 oauthlib==3.2.2 \
     --hash=sha256:8139f29aac13e25d502680e9e19963e83f16838d48a0d71c287fe40e7067fbca \
     --hash=sha256:9859c40929662bec5d64f34d01c99e093149682a3f38915dc0655d5a633dd918
-openai==1.59.9 \
-    --hash=sha256:61a0608a1313c08ddf92fe793b6dbd1630675a1fe3866b2f96447ce30050c448 \
-    --hash=sha256:ec1a20b0351b4c3e65c6292db71d8233515437c6065efd4fd50edeb55df5f5d2
+openai==1.60.0 \
+    --hash=sha256:7fa536cd4b644718645b874d2706e36dbbef38b327e42ca0623275da347ee1a9 \
+    --hash=sha256:df06c43be8018274980ac363da07d4b417bd835ead1c66e14396f6f15a0d5dda
 openpyxl==3.1.5 \
     --hash=sha256:5282c12b107bffeef825f4617dc029afaf41d0ea60823bbb665ef3079dc79de2 \
     --hash=sha256:cf0e3cf56142039133628b5acffe8ef0c12bc902d2aadd3e0fe5878dc08d1050
@@ -829,7 +829,7 @@ python-json-logger==3.2.1 \
 pytz==2024.2 \
     --hash=sha256:2aa355083c50a0f93fa581709deac0c9ad65cca8a9e9beac660adcbd493c798a \
     --hash=sha256:31c7c1817eb7fae7ca4b8c7ee50c72f93aa2dd863de768e1ef4245d426aa0725
-pywin32==308 ; sys_platform == 'win32' \
+pywin32==308 ; (platform_python_implementation != 'PyPy' and sys_platform == 'win32') or platform_system == 'Windows' \
     --hash=sha256:00b3e11ef09ede56c6a43c71f2d31857cf7c54b0ab6e78ac659497abd2834f47 \
     --hash=sha256:1c44539a37a5b7b21d02ab34e6a4d314e0788f1690d65b48e9b0b89f31abbbed \
     --hash=sha256:587f3e19696f4bf96fde9d8a57cec74a57021ad5f204c9e627e15c33ff568897 \
@@ -895,9 +895,9 @@ pyzmq==26.2.0 \
     --hash=sha256:ded0fc7d90fe93ae0b18059930086c51e640cdd3baebdc783a695c77f123dcd9 \
     --hash=sha256:e3e0210287329272539eea617830a6a28161fbbd8a3271bf4150ae3e58c5d0e6 \
     --hash=sha256:ea7f69de383cb47522c9c208aec6dd17697db7875a4674c4af3f8cfdac0bdeae
-referencing==0.35.1 \
-    --hash=sha256:25b42124a6c8b632a425174f24087783efb348a6f1e0008e63cd4466fedf703c \
-    --hash=sha256:eda6d3234d62814d1c64e305c1331c9a3a6132da475ab6382eaa997b21ee75de
+referencing==0.36.1 \
+    --hash=sha256:363d9c65f080d0d70bc41c721dce3c7f3e77fc09f269cd5c8813da18069a6794 \
+    --hash=sha256:ca2e6492769e3602957e9b831b94211599d2aade9477f5d44110d2530cf9aade
 regex==2024.11.6 \
     --hash=sha256:086a27a0b4ca227941700e0b31425e7a28ef1ae8e5e05a33826e17e47fbfdba0 \
     --hash=sha256:0a86e7eeca091c09e021db8eb72d54751e527fa47b8d5787caf96d9831bd02ad \
diff --git a/text_2_sql/GETTING_STARTED.md b/text_2_sql/GETTING_STARTED.md
@@ -5,13 +5,13 @@ To get started, perform the following steps:
 **Execute the following commands in the `deploy_ai_search_indexes` directory:**
 
 1. Setup Azure OpenAI in your subscription with **gpt-4o-mini** & an embedding model, alongside a SQL Server sample database, AI Search and a storage account.
-2. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
-3. Clone this repository and deploy the AI Search text2sql indexes from `deploy_ai_search_indexes`. See the instructions in the **Steps for Text2SQL Index Deployment (For Structured RAG)** section of the `deploy_ai_search_indexes/README.md`.
+2. Clone this repository and deploy the AI Search text2sql indexes from `deploy_ai_search_indexes`. See the instructions in the **Steps for Text2SQL Index Deployment (For Structured RAG)** section of the `deploy_ai_search_indexes/README.md`.
+3. Create your `.env` file based on the provided sample `deploy_ai_search_indexes/.env.example`. Place this file in the same place in `deploy_ai_search_indexes/.env`.
 
 **Execute the following commands in the `text_2_sql_core` directory:**
 
 4. Create your `.env` file based on the provided sample `text_2_sql/.env.example`. Place this file in the same place in `text_2_sql/.env`.
-5. Run `uv sync` within the text_2_sql directory to install dependencies.
+5. Run `uv sync` within the `text_2_sql_core` directory to install dependencies.
     - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
     - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
 6. Create your `.env` file based on the provided sample `text_2_sql/.env.example`. Place this file in the same place in `text_2_sql/.env`.
@@ -20,7 +20,7 @@ To get started, perform the following steps:
 
 **Execute the following commands in the `autogen` directory:**
 
-9. Run `uv sync` within the text_2_sql directory to install dependencies.
+9. Run `uv sync` within the `autogen` directory to install dependencies.
     - Install the optional dependencies if you need a database connector other than TSQL. `uv sync --extra <DATABASE ENGINE>`
     - See the supported connectors in `text_2_sql_core/src/text_2_sql_core/connectors`.
 10. Navigate to `autogen` directory to view the AutoGen implementation. Follow the steps in `Iteration 5 - Agentic Vector Based Text2SQL.ipynb` to get started.
diff --git a/uv.lock b/uv.lock
