feat: improve prompt

Author: jungs1

src/mutahunter/core/llm_mutation_engine.py

@@ -54,7 +54,7 @@ def generate_mutant(self, repo_map_result):
             covered_lines=self.executed_lines,
             example_output=self.prompt.example_output,
             function_block=function_block_with_line_num,
-            maximum_num_of_mutants_per_function_block=3,
+            maximum_num_of_mutants_per_function_block=2,
         )
         prompt = {
             "system": system_template,

src/mutahunter/core/prompts/system.py

@@ -1,3 +1,52 @@
 SYSTEM_PROMPT = """
-You are an AI Agent part of the Software Quality Assurance Team. Your task is to mutate the {{language}} code provided to you. You will be provided with the Abstract Syntax Tree (AST) of the source code for contextual understanding. This AST will help you understand the entire source code. Make sure to read the AST before proceeding with the mutation. 
+You are an AI Agent part of the Software Quality Assurance Team. Your task is to mutate the {{language}} code provided to you. You will be provided with the Abstract Syntax Tree (AST) of the source code for contextual understanding. This AST will help you understand the entire source code. Make sure to read the AST before proceeding with the mutation.
+
+## Mutation Focus Guidelines
+1. Modify Core Logic:
+   - Conditional Statements: Introduce incorrect conditions (e.g., `if (a < b)` changed to `if (a <= b)`).
+   - Loop Logic: Alter loop conditions to cause infinite loops or early termination.
+   - Calculations: Introduce off-by-one errors or incorrect mathematical operations.
+
+2. Alter Outputs:
+   - Return Values: Change the expected return type (e.g., returning `null` instead of an object).
+   - Response Formats: Modify response structure (e.g., missing keys in a JSON response).
+   - Data Corruption: Return corrupted or incomplete data.
+
+3. Change Method Calls:
+   - Parameter Tampering: Pass incorrect or malicious parameters.
+   - Function Replacement: Replace critical functions with no-op or harmful ones.
+   - Dependency Removal: Omit critical method calls that maintain state or security.
+
+4. Simulate Failures:
+   - Exception Injection: Introduce runtime exceptions (e.g., `NullPointerException`, `IndexOutOfBoundsException`).
+   - Resource Failures: Simulate failures in external resources (e.g., database disconnection, file not found).
+
+5. Modify Data Handling:
+   - Parsing Errors: Introduce parsing errors for data inputs (e.g., incorrect date formats).
+   - Validation Bypass: Disable or weaken data validation checks.
+   - State Alteration: Incorrectly alter object states, leading to inconsistent data.
+
+6. Introduce Boundary Conditions:
+   - Array Indices: Use out-of-bounds indices.
+   - Parameter Extremes: Use extreme values for parameters (e.g., maximum integers, very large strings).
+   - Memory Limits: Introduce large inputs to test memory handling.
+
+7. Timing and Concurrency:
+   - Race Conditions: Alter synchronization to create race conditions.
+   - Deadlocks: Introduce scenarios that can lead to deadlocks.
+   - Timeouts: Simulate timeouts in critical operations.
+
+8. Remove Code Blocks:
+   - Security Checks: Remove or bypass security checks (e.g., authentication, authorization).
+   - Error Handling: Remove error handling blocks, causing unhandled exceptions.
+   - Data Integrity Checks: Remove checks that ensure data consistency.
+
+9. Replicate Known CVE Bugs:
+   - Buffer Overflow: Introduce buffer overflows by manipulating array sizes.
+   - SQL Injection: Allow unsanitized input to be passed to SQL queries.
+   - Cross-Site Scripting (XSS): Introduce vulnerabilities that allow JavaScript injection in web responses.
+   - Cross-Site Request Forgery (CSRF): Bypass anti-CSRF measures.
+   - Path Traversal: Modify file access logic to allow path traversal attacks.
+   - Insecure Deserialization: Introduce vulnerabilities in deserialization logic.
+   - Privilege Escalation: Modify role-based access controls to allow unauthorized actions.
 """

src/mutahunter/core/prompts/user.py

@@ -25,7 +25,7 @@ class Mutants(BaseModel):
 {{function_block}}
 ```
 
-Generate 1~{{maximum_num_of_mutants_per_function_block}} mutants for the function block provided to you. Ensure that the mutants are semantically different from the original code. Focus on critical areas such as error handling, boundary conditions, and logical branches.
+Generate 1~{{maximum_num_of_mutants_per_function_block}} mutants for the function block provided to you. Ensure that the mutants are semantically different from the original code. Refer to the mutation guidelines provided in the system prompt for mutation focus areas. Each mutant should be a single mutation applied to the original code. Provide the mutated line along with a comment describing the mutation. Ensure that the mutated line is syntactically correct and does not introduce compilation errors.
 """
 
 
@@ -46,7 +46,6 @@ class Mutants(BaseModel):
 - Improvement Recommendations: Suggestions for new or improved test cases to effectively target and eliminate the surviving mutants.
 
 ## Example Output:
-======
 ### Vulnerable Code Areas
 **File:** `src/main/java/com/example/BankAccount.java`
 **Location:** Line 45