|
170 | 170 | "id": "b7df2c30-a498-4661-a37e-ae19741fa6ba",
|
171 | 171 | "metadata": {},
|
172 | 172 | "source": [
|
173 |
| - "### Temperature Tuning" |
174 |
| - ] |
175 |
| - }, |
176 |
| - { |
177 |
| - "cell_type": "markdown", |
178 |
| - "id": "ffdb3f34-042e-4107-a1cd-afaaadc8e9c0", |
179 |
| - "metadata": {}, |
180 |
| - "source": [ |
| 173 | + "### Temperature Tuning\n", |
| 174 | + "\n", |
181 | 175 | "The `temperature` parameter in LLMs plays a pivotal role in determining the predictability of the output. \n",
|
| 176 | + "\n", |
182 | 177 | "- **Lower temperature values (e.g., 0.2)** lead to more predictable and consistent responses, but may risk being overly constrained or repetitive.\n",
|
| 178 | + "\n", |
183 | 179 | "- **Higher temperature values (e.g., 1.0)** introduce more randomness and diversity, but can result in less consistency and occasional incoherence.\n",
|
184 | 180 | "The choice of temperature value is a trade-off between consistency and variety, and should be tailored to the specific requirements of your task."
|
185 | 181 | ]
|
|
345 | 341 | "id": "65d80452-c074-4341-ae3d-168aadb913e5",
|
346 | 342 | "metadata": {},
|
347 | 343 | "source": [
|
348 |
| - "In the Python code above leverage on `numpy` library, you can see that\n", |
| 344 | + "In the Python code above leveraging on `numpy` library, you can see that\n", |
| 345 | + "\n", |
349 | 346 | "- `softargmax` is a function that computes the soft(arg)max of an array of logits `x` for a given temperature `T`.\n",
|
350 | 347 | "- We define an array of logits and compute the soft(arg)max for different temperatures.\n",
|
351 | 348 | "- When you run this code, you'll see that as the temperature increases, the soft(arg)max output becomes more uniform (i.e., the probabilities are more evenly distributed), and as the temperature decreases, the soft(arg)max output becomes more peaky (i.e., one probability dominates the others). This illustrates how temperature can control the randomness of the model's output.\n",
|
| 349 | + "\n", |
352 | 350 | "- To close this off, taking the max of the soft(arg)max output, you will observe how it gets more random in the max value as the soft(arg)max output becomes more uniform. This links to the concept of how the next word gets more random because of the max of the uniformity of the soft(arg)max output."
|
353 | 351 | ]
|
354 | 352 | },
|
|
548 | 546 | "id": "328f3191-2721-442b-9006-e36ab3e909b4",
|
549 | 547 | "metadata": {},
|
550 | 548 | "source": [
|
551 |
| - "In the code above,\n", |
| 549 | + "In the code above\n", |
| 550 | + "\n", |
552 | 551 | "- `top_k` is a function that computes the top `k` logits from an array of logits.\n",
|
| 552 | + "\n", |
553 | 553 | "- We define an array of logits and compute the top `k` logits for different values of `k`.\n",
|
| 554 | + "\n", |
554 | 555 | "- When you run this code, you'll see that as `k` increases, more logits are considered. This illustrates how `top_k` can control the number of tokens considered by the model."
|
555 | 556 | ]
|
556 | 557 | },
|
|
581 | 582 | "name": "python",
|
582 | 583 | "nbconvert_exporter": "python",
|
583 | 584 | "pygments_lexer": "ipython3",
|
584 |
| - "version": "3.11.5" |
| 585 | + "version": "3.11.0" |
585 | 586 | }
|
586 | 587 | },
|
587 | 588 | "nbformat": 4,
|
|
0 commit comments