Skip to content

add support for thinking #521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 29, 2025
+40 −0
Merged

add support for thinking #521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eaad6df
add support for thinking
drifkin May 27, 2025
f8c6cd5
add thinking example to example index
drifkin May 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,6 @@ Requirement: `pip install tqdm`
### Ollama Embed - Generate embeddings with a model
- [embed.py](embed.py)


### Thinking - Enable thinking mode for a model
- [thinking.py](thinking.py)
13 changes: 13 additions & 0 deletions examples/thinking.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from ollama import chat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add this example to the examples/README.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh thank you! I totally didn't see that file


messages = [
{
'role': 'user',
'content': 'What is 10 + 23?',
},
]

response = chat('deepseek-r1', messages=messages, think=True)

print('Thinking:\n========\n\n' + response.message.thinking)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
print('Thinking:\n========\n\n' + response.message.thinking)
print('Thinking:')
print(response.message.thinking)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and can do the same below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ha turns out I did have a version like that before, but I really wanted to highlight that there are two fields now, so having two print lines total feels like it slightly helps with that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool yeah that makes sense - could we just get rid of the === portion then and just do newline?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ===== is very intentional to do a markdown style header that's much more noticeable in its plaintext form, like

Thinking:
========

<Here is the thinking content>

Response:
========

<Here is the response>

print('\nResponse:\n========\n\n' + response.message.content)
12 changes: 12 additions & 0 deletions ollama/_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,7 @@ def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: Literal[False] = False,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand All @@ -283,6 +284,7 @@ def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: Literal[True] = True,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand All @@ -295,6 +297,7 @@ def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: bool = False,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand Down Expand Up @@ -341,6 +344,7 @@ def add_two_numbers(a: int, b: int) -> int:
messages=list(_copy_messages(messages)),
tools=list(_copy_tools(tools)),
stream=stream,
think=think,
format=format,
options=options,
keep_alive=keep_alive,
Expand Down Expand Up @@ -694,6 +698,7 @@ async def generate(
template: str = '',
context: Optional[Sequence[int]] = None,
stream: Literal[False] = False,
think: Optional[bool] = None,
raw: bool = False,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
images: Optional[Sequence[Union[str, bytes, Image]]] = None,
Expand All @@ -712,6 +717,7 @@ async def generate(
template: str = '',
context: Optional[Sequence[int]] = None,
stream: Literal[True] = True,
think: Optional[bool] = None,
raw: bool = False,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
images: Optional[Sequence[Union[str, bytes, Image]]] = None,
Expand All @@ -729,6 +735,7 @@ async def generate(
template: Optional[str] = None,
context: Optional[Sequence[int]] = None,
stream: bool = False,
think: Optional[bool] = None,
raw: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
images: Optional[Sequence[Union[str, bytes, Image]]] = None,
Expand Down Expand Up @@ -756,6 +763,7 @@ async def generate(
template=template,
context=context,
stream=stream,
think=think,
raw=raw,
format=format,
images=list(_copy_images(images)) if images else None,
Expand All @@ -773,6 +781,7 @@ async def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: Literal[False] = False,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand All @@ -786,6 +795,7 @@ async def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: Literal[True] = True,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand All @@ -798,6 +808,7 @@ async def chat(
*,
tools: Optional[Sequence[Union[Mapping[str, Any], Tool, Callable]]] = None,
stream: bool = False,
think: Optional[bool] = None,
format: Optional[Union[Literal['', 'json'], JsonSchemaValue]] = None,
options: Optional[Union[Mapping[str, Any], Options]] = None,
keep_alive: Optional[Union[float, str]] = None,
Expand Down Expand Up @@ -845,6 +856,7 @@ def add_two_numbers(a: int, b: int) -> int:
messages=list(_copy_messages(messages)),
tools=list(_copy_tools(tools)),
stream=stream,
think=think,
format=format,
options=options,
keep_alive=keep_alive,
Expand Down
12 changes: 12 additions & 0 deletions ollama/_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,9 @@ class GenerateRequest(BaseGenerateRequest):
images: Optional[Sequence[Image]] = None
'Image data for multimodal models.'

think: Optional[bool] = None
'Enable thinking mode (for thinking models).'


class BaseGenerateResponse(SubscriptableBaseModel):
model: Optional[str] = None
Expand Down Expand Up @@ -248,6 +251,9 @@ class GenerateResponse(BaseGenerateResponse):
response: str
'Response content. When streaming, this contains a fragment of the response.'

thinking: Optional[str] = None
'Thinking content. Only present when thinking is enabled.'

context: Optional[Sequence[int]] = None
'Tokenized history up to the point of the response.'

Expand All @@ -263,6 +269,9 @@ class Message(SubscriptableBaseModel):
content: Optional[str] = None
'Content of the message. Response messages contains message fragments when streaming.'

thinking: Optional[str] = None
'Thinking content. Only present when thinking is enabled.'

images: Optional[Sequence[Image]] = None
"""
Optional list of image data for multimodal models.
Expand Down Expand Up @@ -345,6 +354,9 @@ def serialize_model(self, nxt):
tools: Optional[Sequence[Tool]] = None
'Tools to use for the chat.'

think: Optional[bool] = None
'Enable thinking mode (for thinking models).'


class ChatResponse(BaseGenerateResponse):
"""
Expand Down
Loading