Skip to content

Empty results on large queries when using fetchall() #44

Open
@gh-dzuluaga

Description

@gh-dzuluaga

Hello there.

I've found that these two examples will not always return the same thing on large queries:

  1. fetchall
cur.execute(large_query)
rows = cur.fetchall()
print(rows)
  1. iterator
for r in cur.execute(large_query):
  print(r)

The fetchall approach ends up returning an empty list every time I use it when running large queries, but the iterator approach prints the result successfully

Does anyone know why this may be happening?

I don't know how I should supply a working example to debug this problem, as my queries all contain domain logic of my company. Nevertheless, I can say that I'm connecting around 8 Postgres databases with 100k rows per table and equal schemas, and I union the tables from all databases together with UNION views, like this:

CREATE OR REPLACE VIEW hive.default."table_unioned_view" AS 
SELECT 'database1' AS database, "other_column" FROM database1.public.table
UNION ALL
SELECT 'database2' AS database, "other_column" FROM database2.public.table
UNION ALL
SELECT 'database3' AS database, "other_column" FROM database3.public.table;

And the big query that I'm running uses these type of views

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions