Skip to content

Conversation

@Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Oct 28, 2025

Related Issue(s)

#1427
#1431
#1432
#1434

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@github-actions
Copy link
Contributor

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1479

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Pouyanpi Pouyanpi force-pushed the docs/bot-thinking-rails branch 2 times, most recently from 0501e6a to aca68bd Compare October 29, 2025 09:28
update

update

simplify

cleanup
@Pouyanpi Pouyanpi force-pushed the docs/bot-thinking-rails branch from aca68bd to 8033392 Compare October 29, 2025 09:31
Add a note specifying that bot reasoning guardrails are supported only
in Colang 1.0. Update example file references for improved clarity.
@Pouyanpi Pouyanpi self-assigned this Oct 29, 2025
@Pouyanpi Pouyanpi added this to the v0.18.0 milestone Oct 29, 2025
@Pouyanpi Pouyanpi added the documentation Improvements or additions to documentation label Oct 29, 2025
@Pouyanpi Pouyanpi marked this pull request as ready for review October 29, 2025 09:44
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR adds documentation for a new feature that allows developers to access and guardrail bot reasoning traces (exposed via the bot_thinking variable). The documentation covers three access methods - Colang flows, Python actions, and prompt templates - with examples ranging from simple pattern matching to complete self-check output implementations. The guide fits naturally into the advanced user guides section alongside other specialized guardrail features like bot-message-instructions and tools-integration, following the established pattern of documenting complex features with progressive examples and reference implementations.

PR Description Notes:

  • The description, related issues, and checklist are all empty/unchecked. Consider adding a brief explanation of what bot thinking/reasoning guardrails are and why this documentation was added.

Important Files Changed

Filename Score Overview
docs/index.md 1/5 Adds new bot-thinking-guardrails page to TOC but contains critical typo: 'advaced' instead of 'advanced' in file path
docs/user-guides/advanced/bot-thinking-guardrails.md 4/5 Comprehensive new documentation guide explaining bot_thinking variable access patterns with examples and reference implementations

Confidence score: 1/5

  • This PR contains a critical typo that will break the documentation build and must be fixed before merging.
  • Score reflects a single-character typo in the TOC file path ('advaced' vs 'advanced') that will prevent Sphinx/MkDocs from locating the new documentation file, causing build failures.
  • The docs/index.md file requires immediate attention to correct line 71 from user-guides/advaced/bot-thinking-guardrails to user-guides/advanced/bot-thinking-guardrails to match the actual file location.

Sequence Diagram

sequenceDiagram
    participant User
    participant LLMRails
    participant ReasoningLLM as Reasoning LLM<br/>(Main Model)
    participant OutputRails as Output Rails
    participant ColangFlow as Colang Flow<br/>(check_reasoning)
    participant CustomAction as Custom Action<br/>(check_reasoning_quality)
    participant SelfCheckLLM as Self-Check LLM<br/>(Moderation)
    participant PromptTemplate as Prompt Template

    User->>LLMRails: Send user message
    LLMRails->>ReasoningLLM: Generate response with reasoning
    ReasoningLLM-->>LLMRails: Return response + reasoning trace
    LLMRails->>LLMRails: Extract reasoning to $bot_thinking variable
    
    alt Output Rails with Colang Flow
        LLMRails->>OutputRails: Trigger output rails
        OutputRails->>ColangFlow: Execute flow with $bot_thinking
        ColangFlow->>ColangFlow: Check if "confidential" in $bot_thinking
        alt Contains sensitive content
            ColangFlow-->>OutputRails: Block response
            OutputRails-->>LLMRails: bot refuse to respond
        else Safe content
            ColangFlow-->>OutputRails: Allow response
        end
    end
    
    alt Output Rails with Custom Action
        LLMRails->>OutputRails: Trigger output rails
        OutputRails->>CustomAction: execute check_reasoning_quality(context)
        CustomAction->>CustomAction: Get context.get("bot_thinking")
        CustomAction->>CustomAction: Check forbidden patterns
        alt Forbidden pattern found
            CustomAction-->>OutputRails: Return False
            OutputRails-->>LLMRails: bot refuse to respond
        else Safe reasoning
            CustomAction-->>OutputRails: Return True
        end
    end
    
    alt Self-Check Output with Reasoning
        LLMRails->>OutputRails: Trigger self check output
        OutputRails->>PromptTemplate: Render prompt with {{ bot_thinking }}
        PromptTemplate-->>OutputRails: Prompt includes bot_response and bot_thinking
        OutputRails->>SelfCheckLLM: Send prompt for moderation
        SelfCheckLLM->>SelfCheckLLM: Evaluate response and reasoning
        alt Should block
            SelfCheckLLM-->>OutputRails: "Yes"
            OutputRails-->>LLMRails: Block response
        else Should allow
            SelfCheckLLM-->>OutputRails: "No"
            OutputRails-->>LLMRails: Allow response
        end
    end
    
    LLMRails-->>User: Return final response or refusal
Loading

2 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This review covers only the changes made since the last review, not the entire PR. A critical syntax error was introduced on line 10 of docs/index.md where an HTML comment closing tag is malformed (<!-- end-documentation-reuse -- missing the closing >). This will almost certainly break the documentation build process. The previously identified typo on line 71 (path correction from 'advaced' to 'advanced') appears to remain unfixed. No other files were modified in this iteration.

Important Files Changed

Filename Score Overview
docs/index.md 1/5 Introduced a malformed HTML comment tag on line 10 that will likely break documentation builds; typo fix on line 71 may still be pending

Confidence score: 0/5

  • This PR will almost certainly cause immediate documentation build failures and should not be merged in its current state.
  • Score reflects a critical syntax error (malformed HTML comment on line 10) that will break the Sphinx/MkDocs build pipeline, making the documentation generation fail completely.
  • Pay immediate attention to docs/index.md line 10—the comment tag must be properly closed as <!-- end-documentation-reuse --> before this can be merged.

Additional Comments (1)

  1. docs/index.md, line 10 (link)

    syntax: incomplete closing comment tag: missing closing >

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This review covers only the changes made since the last review, not the entire PR. The developer has addressed previous feedback by updating GitHub repository links from commit-specific URLs to branch-based URLs (develop). Specifically, the links on lines 146 and 200 were changed to match the branch-based format already used on line 194, ensuring consistency and maintainability across all three reference links in the documentation. This change ensures that all documentation links point to the current codebase rather than historical snapshots.

Important Files Changed

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 5/5 Updated two GitHub links from commit-specific to branch-based URLs for consistency

Confidence score: 5/5

  • This PR is safe to merge with minimal risk
  • All previous style and consistency issues have been resolved; the changes are purely documentation link updates with no code impact
  • No files require special attention

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@Pouyanpi Pouyanpi requested a review from miyoungc October 29, 2025 10:00
@miyoungc miyoungc mentioned this pull request Oct 30, 2025
4 tasks
@miyoungc
Copy link
Collaborator

Doc edit PR: #1484

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants