feat(vertica-driver): VerticaDriver #7289

timbrownls20 · 2023-10-24T04:19:50Z

Check List

Tests has been run in packages where changes made if available
Linter has been run for changed code
Tests for the changes have been added if not covered yet
Docs have been added / updated if required

Issue Reference this PR resolves

Description of Changes Made (if issue reference is not provided)

New @cubejs-backend/vertica-driver package based on vertica-nodejs -- it includes driver itself and basic tests for overridden BaseDriver methods.

published to npm npm i @knowitall/vertica-driver

continuation of this stale PR #5100

vercel · 2023-10-24T04:19:58Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Skipped Deployments

Name	Status	Preview	Updated (UTC)
examples-angular-dashboard	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-d3	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-dashboard	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-data-table	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-highcharts	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-material-ui	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-react-pivot-table	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am
examples-vue-query-builder	⬜️ Ignored (Inspect)	Visit Preview	Dec 2, 2024 0:50am

paveltiunov · 2023-10-25T03:38:06Z

@timbrownls20 Thanks for contributing! Could you please write a readme on your npm package on how to install and use it? We'll add a backlink to it in the third-party driver list.

timbrownls20 · 2023-10-26T05:46:59Z

@timbrownls20 Thanks for contributing! Could you please write a readme on your npm package on how to install and use it? We'll add a backlink to it in the third-party driver list.

No problem. Added instructions to README for the npm package. Wasn't sure what you wanted, so hope this is OK. If you need amends could you post a link to an example. Many Thanks

codecov · 2023-10-27T19:17:47Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 47.04%. Comparing base (5a27200) to head (ea68485).
Report is 32 commits behind head on master.

❗ Current head ea68485 differs from pull request most recent head 04334c0. Consider uploading reports for the commit 04334c0 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7289      +/-   ##
==========================================
- Coverage   48.02%   47.04%   -0.99%     
==========================================
  Files         154      155       +1     
  Lines       21019    20768     -251     
  Branches     5264     5224      -40     
==========================================
- Hits        10095     9770     -325     
+ Misses      10723    10678      -45     
- Partials      201      320     +119

Flag	Coverage Δ
cube-backend	`47.04% <ø> (-0.99%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

paveltiunov · 2023-10-28T05:23:40Z

@timbrownls20 I actually meant README of @knowitall/vertica-driver. Please see the process here https://github.com/cube-js/cube/blob/master/CONTRIBUTING.md#contributing-database-drivers. You can see https://www.npmjs.com/package/arangodb-cubejs-driver as an example.

timbrownls20 · 2024-11-19T05:28:58Z

@timbrownls20 Can you please run yarn run lint:fix to fix most of the code style warnings that arose? and it would be cool if you fix the rest too. Thnx!

No probs - I've fixed some lint issues to other files within the packages that I'd changed. Hope that's what you meant. Cheers

timbrownls20 · 2024-11-19T05:46:59Z

@timbrownls20 May I kindly ask you to add a page to the docs?

We have a list of supported data sources with linked pages for individual tools, e.g., Postgres. You can find the source code in the same repo: https://github.com/cube-js/cube/tree/master/docs/pages/product/configuration

It would be great if you can add a page for Vertical as well.

You can use this as an icon: https://static.cube.dev/icons/vertica.svg

Hi - docs added. Cheers

KSDaemon · 2024-11-27T11:05:58Z

@timbrownls20 thnx for resolving linter issues! Could you please sync with the upstream? Cause This branch has conflicts that must be resolved. This should be pretty easy! Thnx!

timbrownls20 · 2024-12-02T00:51:32Z

@timbrownls20 thnx for resolving linter issues! Could you please sync with the upstream? Cause This branch has conflicts that must be resolved. This should be pretty easy! Thnx!

No problem - conflicts resolved

KSDaemon · 2024-12-04T11:45:12Z

@timbrownls20 Thnx for the updates! I was just about to merge, but found a few issues:

Some Build/integration tests are failing. I know, it looks like they are not related at all... But they fails. Some investigation leads me to the potentially related issues (azure-sdk-for-js/issues/29014, [Bug] Usage of node: imports for builtin modules jestjs/jest#11637). And I noticed that Vertica is requiring "jest": "^26.6.3". Could you please update to "jest": "^27"? We use it across all the repo in tests. Hoping this will resolve the issue.
I noticed that while the tests itself are ready, they are not triggered in CI. Could you please add them to run in CI here:

Thank you!

….js-korowa into vertica_driver

timbrownls20 · 2024-12-06T00:39:54Z

@timbrownls20 Thnx for the updates! I was just about to merge, but found a few issues:

Some Build/integration tests are failing. I know, it looks like they are not related at all... But they fails. Some investigation leads me to the potentially related issues (azure-sdk-for-js/issues/29014, [Bug] Usage of node: imports for builtin modules jestjs/jest#11637). And I noticed that Vertica is requiring "jest": "^26.6.3". Could you please update to "jest": "^27"? We use it across all the repo in tests. Hoping this will resolve the issue.

I noticed that while the tests itself are ready, they are not triggered in CI. Could you please add them to run in CI here:

https://github.com/cube-js/cube/actions/runs/12131452313/workflow?pr=7289#L26

https://github.com/cube-js/cube/actions/runs/12131452313/workflow?pr=7289#L51

https://github.com/cube-js/cube/actions/runs/12131452313/workflow?pr=7289#L232

https://github.com/cube-js/cube/actions/runs/12131452312/workflow?pr=7289#L320

Thank you!

No problem. I've made those changes. Let me know if you need anything else. All the best

KSDaemon · 2024-12-06T10:26:55Z

@timbrownls20 Thanks for a quick update! That fixes jest failures!

Sadly, the Vertica test fails.

TypeError: Cannot read properties of undefined (reading 'startContainer')
        10 |
        11 |   beforeAll(async () => {
      > 12 |     container = await VerticaDBRunner.startContainer();

I think that might be due to how you use LogWaitStrategy:

import { LogWaitStrategy } from 'testcontainers/build/wait-strategies/log-wait-strategy';

Maybe try to do it like:

.withWaitStrategy(Wait.forLogMessage("Node Status: v_test_node0001: (UP)"))

Here are docs for testcontainers strategies

timbrownls20 · 2024-12-11T00:51:31Z

@timbrownls20 Thanks for a quick update! That fixes jest failures!

Sadly, the Vertica test fails.
TypeError: Cannot read properties of undefined (reading 'startContainer')
        10 |
        11 |   beforeAll(async () => {
      > 12 |     container = await VerticaDBRunner.startContainer();
I think that might be due to how you use LogWaitStrategy:
import { LogWaitStrategy } from 'testcontainers/build/wait-strategies/log-wait-strategy';
Maybe try to do it like:
.withWaitStrategy(Wait.forLogMessage("Node Status: v_test_node0001: (UP)"))
Here are docs for testcontainers strategies

So I think the problem is that the vertica runner doesn't exist in the cube repo (yet) so it can't find it - so I've published a version to our own repo https://github.com/knowitall (terrible name - sorry) so it will work and I can raise another PR to point it back to the cube repo. Hope that makes sense - can you see if that fixes the issue. Thanks

EDIT: Seen test is still failing - looking

KSDaemon · 2024-12-13T11:13:00Z

Still see this error in the integration test :(

error: Node startup/recovery in progress. Not yet ready to accept connections

KSDaemon · 2025-01-06T20:38:07Z

Hey @timbrownls20, Merry Christmas and Happy New Year!
Any updates? Let's nail this down! :)
Feel free to ask, if you need some help!

PolitePp · 2025-01-08T14:16:51Z

Hello, dear @KSDaemon
Can you change this row to export TEST_VERTICA_VERSION=12.0.4-0?

11.x is out of support and has some known issues in Docker.

Also, it's better to update this method like this:

export class VerticaDBRunner extends DbRunnerAbstract {
  public static startContainer() {
    const version = process.env.TEST_VERTICA_VERSION || '12.0.4-0';

    const container = new GenericContainer(`vertica/vertica-ce:${version}`)
      .withEnvironment({ TZ: 'Antarctica/Troll', VERTICA_DB_NAME: 'test', VMART_ETL_SCRIPT: '', VMART_ETL_SQL: '' })
      .withExposedPorts(5433)
      .withStartupTimeout(60 * 1000)
      .withWaitStrategy(
        Wait.forLogMessage('Vertica is now running')
      );

    return container.start();
  }
}

It removes some initiation scripts (because they're not required for tests) and this message shows at the end of the initialization (at this stage a database should be run). I don't think that it requires separate PR and I can't edit the current one. I tested it in my repo and the integration passed successfully.

KSDaemon · 2025-01-08T17:53:45Z

@timbrownls20 Hehey! Thnx for the notes! I can't make edits in your fork, so I pulled your changes to the local branch, and updated as you suggest! And..... It works! Yahoo! Finally!
Here is a 🟢 PR, that I plan to merge soon! Thanks again for the hard work!

korowa and others added 15 commits August 14, 2022 21:27

feat(vertica-driver): VerticaDriver

b50082a

Merge remote-tracking branch 'upstream/master' into vertica_driver

2a5aebb

fixed linter error

f5ecb77

Merge remote-tracking branch 'upstream/master' into vertica_driver

5fbbb1a

Merge remote-tracking branch 'upstream/master' into vertica_driver

ccdd87a

fix dependencies versions

098f884

driver & smoke tests

6818253

Merge remote-tracking branch 'upstream/master' into vertica_driver

3b463b7

vertica driver added to integration tests

9d6f43f

Merge branch 'master' into vertica_driver

40270a4

add vertica to smoke test

2eb00eb

update packages

1d1aa6b

vertica driver published

004e119

vertica driver

7770def

Merge branch 'master' into vertica_driver

0fe9da8

timbrownls20 requested review from a team as code owners October 24, 2023 04:19

github-actions bot added the pr:community Contribution from Cube.js community members. label Oct 24, 2023

timbrownls20 mentioned this pull request Oct 24, 2023

feat(vertica-driver): VerticaDriver #5100

Closed

4 tasks

timbrownls20 changed the title ~~Vertica driver~~ feat(vertica-driver): VerticaDriver Oct 24, 2023

paveltiunov self-assigned this Oct 25, 2023

timbrownls20 added 3 commits October 26, 2023 14:50

update README.md

3daaf38

docker installation instruction

abd02a7

up version

63fe819

Merge branch 'master' into vertica_driver

ea68485

Merge branch 'master' into vertica_driver

79b3ca0

igorlukanin assigned KSDaemon Nov 22, 2024

Merge branch 'master' into vertica_driver

7fa80a6

Merge branch 'master' into vertica_driver

786f846

vercel bot deployed to Preview December 4, 2024 10:55 View deployment

timbrownls20 added 2 commits December 6, 2024 10:26

update jest

adcde01

Merge branch 'vertica_driver' of https://github.com/KnowItAllAus/cube…

2161a47

….js-korowa into vertica_driver

vercel bot deployed to Preview December 6, 2024 00:30 View deployment

timbrownls20 added 2 commits December 6, 2024 10:34

include vertica tests

fa025e8

Merge branch 'master' into vertica_driver

b51e6ae

timbrownls20 added 2 commits December 11, 2024 08:58

update wait strategy and test shared package

a88cac3

Merge branch 'master' into vertica_driver

527f3f1

vercel bot deployed to Preview December 10, 2024 23:32 View deployment

change version of vertica driver

78a2119

Merge branch 'master' into vertica_driver

e1aa6e9

vercel bot had a problem deploying to Preview December 11, 2024 22:15 Failure

KSDaemon mentioned this pull request Jan 8, 2025

feat(vertica-driver): Introduce VerticaDriver #9081

Merged

4 tasks

KSDaemon closed this Jan 8, 2025

feat(vertica-driver): VerticaDriver #7289

feat(vertica-driver): VerticaDriver #7289

Uh oh!

Conversation

timbrownls20 commented Oct 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Oct 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paveltiunov commented Oct 25, 2023

Uh oh!

timbrownls20 commented Oct 26, 2023

Uh oh!

codecov bot commented Oct 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

paveltiunov commented Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timbrownls20 commented Nov 19, 2024

Uh oh!

timbrownls20 commented Nov 19, 2024

Uh oh!

KSDaemon commented Nov 27, 2024

Uh oh!

timbrownls20 commented Dec 2, 2024

Uh oh!

KSDaemon commented Dec 4, 2024

Uh oh!

timbrownls20 commented Dec 6, 2024

Uh oh!

KSDaemon commented Dec 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timbrownls20 commented Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KSDaemon commented Dec 13, 2024

Uh oh!

KSDaemon commented Jan 6, 2025

Uh oh!

PolitePp commented Jan 8, 2025

Uh oh!

KSDaemon commented Jan 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

timbrownls20 commented Oct 24, 2023 •

edited

Loading

vercel bot commented Oct 24, 2023 •

edited

Loading

codecov bot commented Oct 27, 2023 •

edited

Loading

paveltiunov commented Oct 28, 2023 •

edited

Loading

KSDaemon commented Dec 6, 2024 •

edited

Loading

timbrownls20 commented Dec 11, 2024 •

edited

Loading