LLAMA2AI API

A simple ASP.NET Core Web API that interfaces with an LLM model running in a Docker container. This API provides a RESTful interface to interact with LLM models like Llama2, allowing you to send prompts and receive text completions.

Prerequisites

• .NET 8.0 SDK or later
• Docker (for running the LLM container)
• An LLM container running Ollama or compatible API

Project Structure

LLAMA2AI/
├── LLAMA2AI.API/           # ASP.NET Core Web API project
│   ├── Controllers/         # API controllers
│   ├── Models/             # Data models
│   ├── Services/           # Business logic services
│   └── appsettings.json    # Configuration
└── README.md               # This file

Getting Started

1. Run the LLM Container

First, make sure you have your LLM running in a Docker container. For example, if using Ollama:

docker run -d -p 11434:11434 --name ollama ollama/ollama

Then pull and run a model (e.g., Llama2):

docker exec -it ollama ollama pull llama2

2. Build and Run the API

Navigate to the API directory and run:

cd LLAMA2AI.API
dotnet restore
dotnet run

The API will be available at:

• HTTP: http://localhost:5000 or http://localhost:5001
• HTTPS: https://localhost:5001 or https://localhost:5001

3. Configure the LLM Connection

Update appsettings.json if your LLM container is running on a different port or URL:

{
  "LLMSettings": {
    "BaseUrl": "http://localhost:11434",
    "Model": "llama2",
    "TimeoutSeconds": 300
  }
}

API Endpoints

POST /api/llm/completion

Send a prompt to the LLM and receive a completion.

Request Body:

{
  "prompt": "What is the capital of France?",
  "systemPrompt": "You are a helpful assistant.",
  "model": "llama2",
  "stream": false
}

Response:

{
  "response": "The capital of France is Paris.",
  "model": "llama2",
  "done": true,
  "createdAt": "2024-01-01T00:00:00Z"
}

Example using curl:

curl -X POST https://localhost:5001/api/llm/completion \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello, how are you?"}' \
  -k

Example using PowerShell:

$body = @{
    prompt = "What is artificial intelligence?"
} | ConvertTo-Json

Invoke-RestMethod -Uri "http://localhost:5000/api/llm/completion" `
    -Method Post -Body $body -ContentType "application/json"

GET /api/llm/health

Check if the API service is running.

Response:

{
  "status": "healthy",
  "service": "LLAMA2AI API"
}

Swagger UI

When running in Development mode, you can access the Swagger UI at:

• http://localhost:5000/swagger (HTTP)
• https://localhost:5001/swagger (HTTPS)

This provides an interactive interface to test the API endpoints.

Configuration

LLMSettings

• BaseUrl: The base URL of your LLM container (default: http://localhost:11434)
• Model: The default model to use (default: llama2)
• TimeoutSeconds: Request timeout in seconds (default: 300)

Docker Support

You can also run the API in a Docker container:

Build Docker Image

docker build -t llama2ai-api -f Dockerfile .

Run Docker Container

docker run -p 8080:8080 --name llama2ai-api llama2ai-api

Note: Make sure the LLM container is accessible from within the Docker network.

Project Structure

• Controllers/LLMController.cs: Contains the API endpoints
• Services/LLMService.cs: Handles communication with the LLM container
• Models/: Contains the data models for requests and responses
• appsettings.json: Application configuration

Error Handling

The API includes comprehensive error handling:

• 400 Bad Request: Invalid request (e.g., empty prompt)
• 503 Service Unavailable: LLM container is not reachable
• 504 Gateway Timeout: Request to LLM timed out
• 500 Internal Server Error: Unexpected errors

Security Notes

• The API is configured to allow CORS from any origin by default. Update Program.cs to restrict this in production.
• Consider adding authentication/authorization for production use.
• Validate and sanitize inputs to prevent prompt injection attacks.

Troubleshooting

1. Cannot connect to LLM container:

   • Ensure the LLM container is running (docker ps)
   • Check the BaseUrl in appsettings.json
   • Verify network connectivity

2. Timeouts:

   • Increase TimeoutSeconds in appsettings.json
   • Check if the model is loaded in the LLM container

3. Port conflicts:

   • Change the ports in launchSettings.json if needed

License

This project is provided as-is for demonstration purposes.