You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: integrations/google-vertex-ai.md
+16-15Lines changed: 16 additions & 15 deletions
Original file line number
Diff line number
Diff line change
@@ -57,7 +57,8 @@ Once installed, you will have access to various Haystack Generators:
57
57
58
58
To use Vertex AI models, you need to have a Google Cloud Platform account and be logged in using Application Default Credentials (ADCs). For more info see the [official documentation](https://colab.research.google.com/corgiredirector?site=https%3A%2F%2Fcloud.google.com%2Fdocs%2Fauthentication%2Fprovide-credentials-adc).
59
59
60
-
To start using Vertex AI generators in Haystack, you need to set the `project_id` variable to a valid project ID that have enough authorization to use Vertex AI. Find your `project_id` in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI see the [official documentation](https://cloud.google.com/cli).
60
+
To start using Vertex AI generators in Haystack, it is essential that your account has access to a project authorized to use Google Vertex AI endpoints. The `project_id` needed for initialization of Vertex AI generators is set during GCP authentication mentioned above. Additonally, you can also set a different `project_id` by passing it as a variable during initialization of the generator.
61
+
You can find your `project_id` in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI see the [official documentation](https://cloud.google.com/cli).
61
62
62
63
### Gemini API models
63
64
@@ -71,7 +72,7 @@ To use Gemini model for text generation, initialize a `VertexAIGeminiGenerator`
71
72
from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator
result = gemini_generator.run(parts= ["What is assemblage in art?"])
76
77
print(result["replies"][0])
77
78
```
@@ -82,7 +83,7 @@ Assemblage in art refers to the creation of a three-dimensional artwork by combi
82
83
83
84
**Multimodality with `gemini-1.5-flash`**
84
85
85
-
To use `gemini-1.5-flash` model for visual question answering, initialize a `VertexAIGeminiGenerator` with `"gemini-1.5-flash"` and `project_id`. Then, run it with the images as well as the prompt:
86
+
To use `gemini-1.5-flash` model for visual question answering, initialize a `VertexAIGeminiGenerator` with `"gemini-1.5-flash"`. Then, run it with the images as well as the prompt:
result = gemini_generator.run(parts= ["What can you tell me about these robots?", *images])
104
105
for answer in result["replies"]:
105
106
print(answer)
@@ -116,15 +117,15 @@ The fourth image is of Marvin from the 1977 film The Hitchhiker's Guide to the G
116
117
117
118
### PaLM API Models
118
119
119
-
You can leverage PaLM API models `text-bison`, `text-unicorn` and `text-bison-32k` through `VertexAITextGenerator` for task generation. To use PaLM models, initialize a `VertexAITextGenerator` with model name and `project_id`.
120
+
You can leverage PaLM API models `text-bison`, `text-unicorn` and `text-bison-32k` through `VertexAITextGenerator` for task generation. To use PaLM models, initialize a `VertexAITextGenerator` with model name.
120
121
121
122
Here'a an example of using `text-unicorn` model with VertexAITextGenerator to extract information as a JSON file:
122
123
123
124
```python
124
125
from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator
You can leverage Codey API models, `code-bison`, `code-bison-32k` and `code-gecko`, through `VertexAICodeGenerator`for code generation. To use Codey models, initialize a `VertexAICodeGenerator` with model name and `project_id`.
139
+
You can leverage Codey API models, `code-bison`, `code-bison-32k` and `code-gecko`, through `VertexAICodeGenerator`for code generation. To use Codey models, initialize a `VertexAICodeGenerator` with model name.
139
140
140
141
Here'a an example of using `code-bison` model for **code generation**:
141
142
```python
142
143
from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator
image_generator_result = image_generator.run("magazine style, 4k, photorealistic, modern red armchair, natural lighting")
181
182
182
183
## (Optional) Save the generated image
@@ -186,13 +187,13 @@ image.save("output.png")
186
187
187
188
**Image Captioning with `imagetext`**
188
189
189
-
To use generate image captions, initialize a VertexAIImageCaptioner with the `imagetext` model and `project_id`. Then, you can run the VertexAIImageCaptioner with the image that you want to caption:
190
+
To use generate image captions, initialize a VertexAIImageCaptioner with the `imagetext` model. Then, you can run the VertexAIImageCaptioner with the image that you want to caption:
190
191
191
192
```python
192
193
from haystack_integrations.components.generators.google_vertex import VertexAIImageCaptioner
**Visual Question Answering (VQA) with `imagetext`**
203
204
204
-
To answer questions about an image, initialize a VertexAIImageQA with the `imagetext` model and `project_id`. Then, you can run it with the `image` and the `question`:
205
+
To answer questions about an image, initialize a VertexAIImageQA with the `imagetext` model. Then, you can run it with the `image` and the `question`:
205
206
206
207
```python
207
208
from haystack.dataclasses.byte_stream import ByteStream
208
209
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
0 commit comments