Skip to content

Fix postgresql #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions deploy_ai_search_indexes/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ snowflake = [
databricks = [
"text_2_sql_core[databricks]",
]
postgresql = [
"text_2_sql_core[postgresql]",
postgres = [
"text_2_sql_core[postgres]",
]
sqlite = [
"text_2_sql_core[sqlite]",
Expand Down
6 changes: 3 additions & 3 deletions image_processing/src/image_processing/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ beautifulsoup4==4.12.3
blis==0.7.11
bs4==0.0.2
catalogue==2.0.10
certifi==2024.12.14
certifi==2025.1.31
cffi==1.17.1 ; platform_python_implementation != 'PyPy'
charset-normalizer==3.4.1
click==8.1.8
Expand All @@ -38,7 +38,7 @@ fsspec==2024.12.0
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
huggingface-hub==0.28.0
huggingface-hub==0.28.1
idna==3.10
isodate==0.7.2
jinja2==3.1.5
Expand Down Expand Up @@ -74,7 +74,7 @@ pyjwt==2.10.1
pymupdf==1.25.2
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
pytz==2024.2
pytz==2025.1
pywin32==308 ; sys_platform == 'win32'
pyyaml==6.0.2
regex==2024.11.6
Expand Down
16 changes: 8 additions & 8 deletions text_2_sql/.env.example
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Environment variables for Text2SQL
IdentityType=<identityType> # system_assigned or user_assigned or key

Text2Sql__DatabaseEngine=<DatabaseEngine> # TSQL or PostgreSQL or Snowflake or Databricks
Text2Sql__DatabaseEngine=<DatabaseEngine> # TSQL or Postgres or Snowflake or Databricks
Text2Sql__UseQueryCache=<Determines if the Query Cache will be used to speed up query generation. Defaults to True.> # True or False
Text2Sql__PreRunQueryCache=<Determines if the results from the Query Cache will be pre-run to speed up answer generation. Defaults to True.> # True or False
Text2Sql__UseColumnValueStore=<Determines if the Column Value Store will be used for schema selection Defaults to True.> # True or False
Expand All @@ -26,13 +26,13 @@ AIService__AzureSearchOptions__Text2SqlColumnValueStore__Index=<Column value sto
Text2Sql__Tsql__ConnectionString=<Tsql databaseConnectionString if using Tsql Data Source>
Text2Sql__Tsql__Database=<Tsql database if using Tsql Data Source>

# PostgreSQL Specific Connection Details
Text2Sql__Postgresql__ConnectionString=<Postgresql databaseConnectionString if using Postgresql Data Source and a connection string>
Text2Sql__Postgresql__Database=<Postgresql database if using Postgresql Data Source>
Text2Sql__Postgresql__User=<Postgresql user if using Postgresql Data Source and not the connections string>
Text2Sql__Postgresql__Password=<Postgresql password if using Postgresql Data Source and not the connections string>
Text2Sql__Postgresql__ServerHostname=<Postgresql serverHostname if using Postgresql Data Source and not the connections string>
Text2Sql__Postgresql__Port=<Postgresql port if using Postgresql Data Source and not the connections string>
# Postgres Specific Connection Details
Text2Sql__Postgres__ConnectionString=<Postgres databaseConnectionString if using Postgres Data Source and a connection string>
Text2Sql__Postgres__Database=<Postgres database if using Postgres Data Source>
Text2Sql__Postgres__User=<Postgres user if using Postgres Data Source and not the connections string>
Text2Sql__Postgres__Password=<Postgres password if using Postgres Data Source and not the connections string>
Text2Sql__Postgres__ServerHostname=<Postgres serverHostname if using Postgres Data Source and not the connections string>
Text2Sql__Postgres__Port=<Postgres port if using Postgres Data Source and not the connections string>

# Snowflake Specific Connection Details
Text2Sql__Snowflake__User=<snowflakeUser if using Snowflake Data Source>
Expand Down
4 changes: 2 additions & 2 deletions text_2_sql/autogen/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ snowflake = [
databricks = [
"text_2_sql_core[databricks]",
]
postgresql = [
"text_2_sql_core[postgresql]",
postgres = [
"text_2_sql_core[postgres]",
]
sqlite = [
"text_2_sql_core[sqlite]",
Expand Down
2 changes: 1 addition & 1 deletion text_2_sql/data_dictionary/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ The following Databases have pre-built scripts for them:
- **Databricks:** `./text_2_sql_core/data_dictionary/databricks_data_dictionary_creator.py`
- **Snowflake:** `./text_2_sql_core/data_dictionary/snowflake_data_dictionary_creator.py`
- **TSQL:** `./text_2_sql_core/data_dictionary/tsql_data_dictionary_creator.py`
- **PostgreSQL:** `./text_2_sql_core/data_dictionary/postgresql_data_dictionary_creator.py`
- **Postgres:** `./text_2_sql_core/data_dictionary/postgres_data_dictionary_creator.py`

If there is no pre-built script for your database engine, take one of the above as a starting point and adjust it.

Expand Down
2 changes: 1 addition & 1 deletion text_2_sql/text_2_sql_core/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ databricks = [
"databricks-sql-connector>=3.0.1",
"pyarrow>=14.0.2,<17",
]
postgresql = [
postgres = [
"psycopg>=3.2.3",
]
sqlite = [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ def get_database_connector():
from text_2_sql_core.connectors.tsql_sql import TsqlSqlConnector

return TsqlSqlConnector()
elif os.environ["Text2Sql__DatabaseEngine"].upper() == "POSTGRESQL":
from text_2_sql_core.connectors.postgresql_sql import (
PostgresqlSqlConnector,
elif os.environ["Text2Sql__DatabaseEngine"].upper() == "POSTGRES":
from text_2_sql_core.connectors.postgres_sql import (
PostgresSqlConnector,
)

return PostgresqlSqlConnector()
return PostgresSqlConnector()
elif os.environ["Text2Sql__DatabaseEngine"].upper() == "SQLITE":
from text_2_sql_core.connectors.sqlite_sql import SQLiteSqlConnector

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
from text_2_sql_core.utils.database import DatabaseEngine, DatabaseEngineSpecificFields


class PostgresqlSqlConnector(SqlConnector):
class PostgresSqlConnector(SqlConnector):
def __init__(self):
super().__init__()

self.database_engine = DatabaseEngine.POSTGRESQL
self.database_engine = DatabaseEngine.POSTGRES

@property
def engine_specific_rules(self) -> str:
Expand Down Expand Up @@ -54,7 +54,7 @@ async def query_execution(
cast_to: any = None,
limit=None,
) -> list[dict]:
"""Run the SQL query against the PostgreSQL database asynchronously.
"""Run the SQL query against the Postgres database asynchronously.

Args:
----
Expand All @@ -67,10 +67,10 @@ async def query_execution(
logging.info(f"Running query: {sql_query}")
results = []

if "Text2Sql__Postgresql__ConnectionString" in os.environ:
logging.info("Postgresql Connection string found in environment variables.")
if "Text2Sql__Postgres__ConnectionString" in os.environ:
logging.info("Postgres Connection string found in environment variables.")

p = urlparse(os.environ["Text2Sql__Postgresql__ConnectionString"])
p = urlparse(os.environ["Text2Sql__Postgres__ConnectionString"])

postgres_connections = {
"dbname": p.path[1:],
Expand All @@ -81,17 +81,17 @@ async def query_execution(
}
else:
logging.warning(
"Postgresql Connection string not found in environment variables. Using individual variables."
"Postgres Connection string not found in environment variables. Using individual variables."
)
postgres_connections = {
"dbname": os.environ["Text2Sql__Postgresql__Database"],
"user": os.environ["Text2Sql__Postgresql__User"],
"password": os.environ["Text2Sql__Postgresql__Password"],
"port": os.environ["Text2Sql__Postgresql__Port"],
"host": os.environ["Text2Sql__Postgresql__ServerHostname"],
"dbname": os.environ["Text2Sql__Postgres__Database"],
"user": os.environ["Text2Sql__Postgres__User"],
"password": os.environ["Text2Sql__Postgres__Password"],
"port": os.environ["Text2Sql__Postgres__Port"],
"host": os.environ["Text2Sql__Postgres__ServerHostname"],
}

# Establish an asynchronous connection to the PostgreSQL database
# Establish an asynchronous connection to the Postgres database
async with await psycopg.AsyncConnection.connect(
**postgres_connections
) as conn:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,12 @@ def create(
data_dictionary_creator = TsqlDataDictionaryCreator(
**kwargs,
)
elif engine == DatabaseEngine.POSTGRESQL:
from text_2_sql_core.data_dictionary.postgresql_data_dictionary_creator import (
PostgresqlDataDictionaryCreator,
elif engine == DatabaseEngine.POSTGRES:
from text_2_sql_core.data_dictionary.postgres_data_dictionary_creator import (
PostgresDataDictionaryCreator,
)

data_dictionary_creator = PostgresqlDataDictionaryCreator(
data_dictionary_creator = PostgresDataDictionaryCreator(
**kwargs,
)
else:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,23 @@
import os

from text_2_sql_core.utils.database import DatabaseEngine
from text_2_sql_core.connectors.postgresql_sql import PostgresqlSqlConnector
from text_2_sql_core.connectors.postgres_sql import PostgresSqlConnector


class PostgresqlDataDictionaryCreator(DataDictionaryCreator):
class PostgresDataDictionaryCreator(DataDictionaryCreator):
def __init__(self, **kwargs):
"""A method to initialize the DataDictionaryCreator class."""
excluded_schemas = ["information_schema", "pg_catalog"]
super().__init__(excluded_schemas=excluded_schemas, **kwargs)

self.database = os.environ["Text2Sql__Postgresql__Database"]
self.database_engine = DatabaseEngine.POSTGRESQL
self.database = os.environ["Text2Sql__Postgres__Database"]
self.database_engine = DatabaseEngine.POSTGRES

self.sql_connector = PostgresqlSqlConnector()
self.sql_connector = PostgresSqlConnector()

@property
def extract_table_entities_sql_query(self) -> str:
"""A property to extract table entities from a PostgreSQL database."""
"""A property to extract table entities from a Postgres database."""
return """SELECT
t.table_name AS "Entity",
t.table_schema AS "EntitySchema",
Expand All @@ -46,7 +46,7 @@ def extract_table_entities_sql_query(self) -> str:

@property
def extract_view_entities_sql_query(self) -> str:
"""A property to extract view entities from a PostgreSQL database."""
"""A property to extract view entities from a Postgres database."""
return """SELECT
v.table_name AS "Entity",
v.table_schema AS "EntitySchema",
Expand All @@ -65,7 +65,7 @@ def extract_view_entities_sql_query(self) -> str:
"EntitySchema", "Entity";"""

def extract_columns_sql_query(self, entity: EntityItem) -> str:
"""A property to extract column information from a PostgreSQL database."""
"""A property to extract column information from a Postgres database."""
return f"""SELECT
c.attname AS "Name",
t.typname AS "DataType",
Expand All @@ -89,7 +89,7 @@ def extract_columns_sql_query(self, entity: EntityItem) -> str:

@property
def extract_entity_relationships_sql_query(self) -> str:
"""A property to extract entity relationships from a PostgreSQL database."""
"""A property to extract entity relationships from a Postgres database."""
return """SELECT
fk_schema.nspname AS "EntitySchema",
fk_tab.relname AS "Entity",
Expand All @@ -116,5 +116,5 @@ def extract_entity_relationships_sql_query(self) -> str:


if __name__ == "__main__":
data_dictionary_creator = PostgresqlDataDictionaryCreator()
data_dictionary_creator = PostgresDataDictionaryCreator()
asyncio.run(data_dictionary_creator.create_data_dictionary())
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ class DatabaseEngine(StrEnum):
DATABRICKS = "DATABRICKS"
SNOWFLAKE = "SNOWFLAKE"
TSQL = "TSQL"
POSTGRESQL = "POSTGRESQL"
POSTGRES = "POSTGRES"
SQLITE = "SQLITE"


Expand Down
Loading
Loading