[Enhancement] Optimize Resource Creation: Eliminate "Create then Search" Pattern

[bsimonWallix]

Issue Summary

The current Terraform provider implementation uses an inefficient "Create then Search" pattern for resource creation, resulting in unnecessary API calls and potential race conditions. This issue proposes a comprehensive refactoring to optimize performance and improve reliability.

Current Problem

Pattern Analysis

Currently, 23 resources follow this inefficient pattern:

1. Pre-check: searchResource*() to verify resource doesn't exist
2. Create: addResource*() with POST request (no ID returned)
3. Search: searchResource*() again to retrieve the created resource ID
4. Set ID: Assign ID to Terraform resource data

Code Example (Current Implementation)

func resourceXXXCreate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
    // 1. Check if resource already exists
    _, ex, err := searchResourceXXX(ctx, d.Get("name").(string), m)
    if err != nil || ex {
        return diag.FromErr(err)
    }
    
    // 2. Create resource (POST without returning ID)
    err = addXXX(ctx, d, m)
    if err != nil {
        return diag.FromErr(err)
    }
    
    // 3. Search again to get the ID
    id, ex, err := searchResourceXXX(ctx, d.Get("name").(string), m)
    if err != nil || !ex {
        return diag.FromErr(err)
    }
    
    d.SetId(id)
    return resourceXXXRead(ctx, d, m)
}

Affected Resources

All these resources use the inefficient pattern:

• Auth Domains: resource_authdomain_ad.go, resource_authdomain_ldap.go, resource_authdomain_saml.go, resource_authdomain_azuread.go
• Devices: resource_device.go, resource_device_localdomain.go, resource_device_service.go
• Domains: resource_domain.go, resource_domain_account.go
• External Auth: resource_externalauth_kerberos.go, resource_externalauth_ldap.go, resource_externalauth_radius.go, resource_externalauth_saml.go, resource_externalauth_tacacs.go
• Users & Groups: resource_user.go, resource_usergroup.go, resource_targetgroup.go, resource_profile.go
• Security: resource_authorization.go
• Time Management: resource_timeframe.go
• Credentials: resource_*_credential.go files

Impact & Issues

Performance Issues

• 3x API calls: Instead of 1 optimized call
• Network latency: Triple the round-trip time
• API load: Unnecessary burden on Bastion servers
• Slow provisioning: Especially for large Terraform configurations

Reliability Issues

• Race conditions: Between resource creation and search
• Intermittent failures: Resource created but not found in search
• Inconsistent state: Potential for orphaned resources

Maintenance Issues

• Code duplication: Same pattern repeated across 23 resources
• Complex debugging: Multiple failure points per creation
• Test complexity: Harder to write reliable integration tests

Proposed Solution

Architecture Overview

bastion/
├── client/
│   ├── client.go           # Enhanced HTTP client
│   ├── session.go          # Session management
│   └── response.go         # Response parsing with ID extraction
├── resources/
│   ├── base/
│   │   ├── crud.go         # Generic CRUD interface
│   │   ├── create.go       # Optimized creation logic
│   │   └── search.go       # Fallback search logic
│   ├── authorization.go    # Individual resources
│   ├── authdomain.go
│   └── ...

Key Improvements

1. Enhanced HTTP Client with ID Extraction

type APIResponse struct {
    Body       string
    StatusCode int
    Headers    http.Header
    ResourceID string // Extracted from response
}

func (c *Client) DoRequestWithIDExtraction(ctx context.Context, method, uri string, body interface{}) (*APIResponse, error) {
    // Enhanced client that attempts to extract resource ID from:
    // - Location header
    // - Response body JSON
    // - Custom headers (X-Resource-ID)
}

2. Generic CRUD Framework

func OptimizedCreate(ctx context.Context, d *schema.ResourceData, m interface{}, config ResourceConfig, ops CRUDOperations) diag.Diagnostics {
    // 1. Pre-existence check
    // 2. Create with ID extraction attempt
    // 3. Fallback to search if extraction fails
    // 4. Set ID and read resource
}

3. Resource-Specific Implementation

func resourceAuthorizationCreate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
    crud := &AuthorizationCRUD{}
    return base.OptimizedCreate(ctx, d, m, authorizationConfig, crud)
}

Expected Benefits

Performance Improvements

• -66% API calls: From 3 calls to 1 call per creation
• -50% latency: Eliminate post-creation search
• +100% throughput: Reduced API load allows more concurrent operations

Reliability Improvements

• Eliminate race conditions: Single atomic operation
• Better error handling: Centralized error management
• Consistent behavior: Unified creation pattern

Maintenance Improvements

• DRY principle: Single implementation for all resources
• Easier testing: Centralized test patterns
• Better debugging: Clear separation of concerns

Implementation Plan

Phase 1: Infrastructure

• Create enhanced bastion/client/ package
• Implement DoRequestWithIDExtraction() method
• Add comprehensive unit tests for ID extraction
• Test against various API response formats

Phase 2: CRUD Framework

• Create bastion/resources/base/ package
• Implement generic OptimizedCreate() function
• Add interface definitions for resource operations
• Create integration tests for the framework

Phase 3: Progressive Migration

• Week 5: Migrate authorization + auth domains (5 resources)
• Week 6: Migrate device-related resources (6 resources)
• Week 7: Migrate external auth resources (5 resources)
• Week 8: Migrate remaining resources (7 resources)
• Add regression tests for each migrated resource
• Performance benchmarks for before/after comparison

Phase 4: Cleanup

• Remove deprecated functions
• Update documentation
• Final performance validation
• Release preparation

Testing Strategy

Unit Tests

• ID extraction from various response formats
• Fallback mechanisms
• Error handling scenarios

Integration Tests

• End-to-end resource creation
• Performance benchmarks
• Concurrent creation scenarios

Regression Tests

• Existing functionality preservation
• Backward compatibility validation
• Edge case handling

Success Metrics

Performance Metrics

• API call reduction: Target 66% fewer calls
• Latency improvement: Target 50% faster creation
• Throughput increase: Measure concurrent creation capacity

Reliability Metrics

• Error rate reduction: Target 90% fewer race condition errors
• Success rate: Target 99.9% successful resource creation
• Consistency: Eliminate intermittent failures

Backward Compatibility

• Gradual migration: Resources migrated one by one
• Fallback support: Search method available if ID extraction fails
• Configuration preservation: No changes to Terraform configurations
• API compatibility: No breaking changes to provider interface

Development Notes

Technical Considerations

• Test ID extraction against actual Wallix Bastion API responses
• Ensure session management compatibility
• Handle API version differences gracefully
• Maintain existing error message formats for user experience

Risk Mitigation

• Feature flags: Enable/disable optimization per resource type
• Comprehensive testing: Before production deployment
• Rollback plan: Quick revert to original implementation
• Monitoring: Track creation success rates post-deployment

---

Priority: High
Complexity: Medium
Impact: High
Type: Enhancement

This optimization represents a significant improvement in provider efficiency and reliability, directly benefiting users with faster Terraform operations and more stable infrastructure management.