docs: update documentation and refactor integration config handling

.gitignore

@@ -38,4 +38,4 @@ profile.cov
 config.json
 
 # Avoid Committing Integration testing configuration file
-integration_servers.json
+integration_config.json

CODE_OF_CONDUCT.md

@@ -0,0 +1,3 @@
+# Code of Conduct
+
+By participating in this project, you agree to comply with the [OBA Code of Conduct](https://github.com/OneBusAway/onebusaway/blob/master/CODE_OF_CONDUCT.md).

CONTRIBUTING.md

@@ -0,0 +1,43 @@
+# Contributing to Watchdog
+
+Please note that this project is released with a [Code of Conduct](./CODE_OF_CONDUCT.md); by participating, you agree to uphold its terms.
+
+## How to Contribute
+
+1. Fork the repository by clicking the fork button in the top-right corner.
+2. Clone your fork locally and set the upstream to the official Watchdog repository.
+3. Follow the setup instructions [here](./README.md#setup).
+4. Keep changes scoped:
+   - Use a dedicated branch per bug fix or feature.
+   - This allows you to open and manage multiple pull requests in parallel.
+5. Stay up to date:
+   - Sync your fork regularly with the upstream `main` branch to avoid conflicts.
+6. **Open an issue first** for new features or significant changes so we can discuss design and direction.
+7. Follow [code guidelines](#code-guidelines) and write conventional commit messages.
+8. Submit a Pull Request
+   - Clearly describe the problem being solved.
+   - Reference related issues in your commits or PR description (e.g., `Fixes #42: Improve GTFS parsing`).
+
+## Code Guidelines
+
+- Run Watchdog using one of the methods described [here](./README.md#running) and ensure there are no build or runtime errors.
+
+- **Formatting**: Run the Go formatter before committing.
+
+```bash
+  go fmt ./...
+```
+
+- **Linting**: Run `go vet` to catch common mistakes.
+
+```bash
+  go vet ./...
+```
+
+- **Testing**:
+  - Write unit tests for new functionality.
+  - Ensure the full test suite passes with:
+
+```bash
+  go test ./...
+```

README.md

@@ -1,26 +1,27 @@
-# watchdog
+# Watchdog
 
 [![Coverage Status](https://coveralls.io/repos/github/OneBusAway/watchdog/badge.svg?branch=main)](https://coveralls.io/github/OneBusAway/watchdog?branch=main)
 
-Golang-based Watchdog service for OBA REST API servers, providing deep observability by exposing a rich suite of Prometheus metrics. These metrics enable comprehensive monitoring of API uptime, GTFS Static and GTFS-RT data integrity, vehicle telemetry, agency and stop coverage, and overall operational health.
+**Watchdog** is a Go-based service that monitors [OneBusAway (OBA)](https://onebusaway.org/) REST API servers.
+It exposes a comprehensive set of **Prometheus metrics** for monitoring:
 
-You can find documentation for the currently exposed metrics along with an interpretation guide [here](./docs/METRICS.md).
+- GTFS Static and GTFS-RT data integrity
+- Vehicle telemetry
+- Agency and stop coverage
+- Overall operational health
+  See the full list of metrics and interpretation guide [here](./docs/METRICS.md)
 
-# Requirements
+## Requirements
 
-Go 1.23 or higher
+- **Go 1.23+**
 
-## Configuration
+## Setup
 
-The watchdog service can be configured using either:
+### Configuration
 
-- A **local JSON configuration file** (`--config-file`).
-- A **remote JSON configuration URL** (`--config-url`).
-  - Using a remote configuration allows you to dynamically add, remove, or update server configurations at runtime.
+Watchdog requires a configuration file (`config.json`) before running. Even placeholder data is necessary to start the service.
 
-### JSON Configuration Format
-
-The JSON configuration file should contain an array of `ObaServer` objects, each representing an OBA server to be monitored. Example:
+#### Example `config.json`
 
 ```json
 [
@@ -35,110 +36,138 @@ The JSON configuration file should contain an array of `ObaServer` objects, each
     "gtfs_rt_api_key": "api-key-1",
     "gtfs_rt_api_value": "api-value-1",
     "agency_id": "agency-1"
-  },
-  {
-    "name": "Test Server 2",
-    "id": 2,
-    "oba_base_url": "https://test2.example.com",
-    "oba_api_key": "test-key-2",
-    "gtfs_url": "https://gtfs2.example.com",
-    "trip_update_url": "https://trip2.example.com",
-    "vehicle_position_url": "https://vehicle2.example.com",
-    "gtfs_rt_api_key": "api-key-2",
-    "gtfs_rt_api_value": "api-value-2",
-    "agency_id": "agency-2"
   }
 ]
 ```
 
-### Local Configuration Setup
+#### Ways to Provide the Config File
+
+#### 1. Local Configuration (recommended for development)
+
+- Copy or rename `config.json.template` → `config.json`
+- Fill in your server values
+- Run with:
+
+```bash
+go run ./cmd/watchdog/ --config-file path/to/config.json
+```
 
-1. Either copy or rename `config.json.template` to `config.json` in the same folder.
-2. Update `config.json` with your OBA server values.
+Note:
 
-Note: The `config.json` file is ignored by Git to prevent accidental commits of sensitive configuration data.
+- ⚠️The file **must** be named `config.json`
+- `config.json` is Git-ignored (to protect secrets)
 
-### Remote Configuration Setup
+#### 2. Remote Configuration (recommended for production)
 
-1. Create a `config.json` file based on the `config.json.template` format.
-2. Fill in your OBA server values in `config.json`.
-3. Host it publicly and point the app to its URL.
+- Prepare `config.json` as above
+- Host it publicly (or on a private server)
+- Run with:
 
-## Sentry Configuration
+```bash
+go run ./cmd/watchdog/ --config-url http://example.com/config.json
+```
 
-To enable Sentry error tracking, set the `SENTRY_DSN` environment variable with your Sentry DSN.
+If authentication is required, set:
 
-```sh
-export SENTRY_DSN="your_sentry_dsn"
+```bash
+export CONFIG_AUTH_USER="username"
+export CONFIG_AUTH_PASS="password"
 ```
 
-# Running
+### Application Options
+
+- **Fetch Interval** → default `30s` (`--fetch-interval <seconds>`)
+- **Environment** → `development` (default), `staging`, `production` (`--env <value>`)
+- **Port** → default `4000` (`--port <number>`)
+
+⚠️ If running with **Docker Compose**, Prometheus runs on `9090` and Grafana on `3000`. Don’t use those ports.
+
+### Environment Variables
 
-#### **Using a Local Configuration File**
+- **Sentry DSN**
 
 ```bash
-go run ./cmd/watchdog/ --config-file ./config.json
+    export SENTRY_DSN="your_sentry_dsn"
 ```
 
-## **Using a Remote Configuration URL with Authentication**
+- **Config Auth (for remote configs)**
 
-To load the configuration from a remote URL that requires basic authentication, follow these steps:
+```bash
+    export CONFIG_AUTH_USER="username"
+    export CONFIG_AUTH_PASS="password"
+```
 
-### 1. **Set the Required Environment Variables**
+## Running
 
-Before running the application, set the `CONFIG_AUTH_USER` and `CONFIG_AUTH_PASS` environment variables with the username and password for authentication.
+It may take a few minutes for Watchdog to start exposing data to Prometheus, since initial setup includes tasks such as downloading the GTFS bundle.
 
-#### On Linux/macOS:
+### 1. Docker Compose (recommended)
+
+Run Watchdog with **Prometheus + Grafana**:
 
 ```bash
-export CONFIG_AUTH_USER="your_username"
-export CONFIG_AUTH_PASS="your_password"
+docker compose up --build
 ```
 
-#### On Windows
+Services:
+
+- Watchdog → `4000`
+- Prometheus → `9090`
+- Grafana → `3000`
+
+Stop services:
 
 ```bash
-set CONFIG_AUTH_USER=your_username
-set CONFIG_AUTH_PASS=your_password
+docker compose down
 ```
 
-#### Run the Application with the Remote URL
+Restart services:
+
+```bash
+docker compose restart
+```
+
+Grafana auto-loads a Go runtime dashboard. Prometheus is pre-configured to scrape Watchdog.
+
+See [Endpoints](#endpoints) to access metrics, health checks, Grafana, and Prometheus.
+
+### 2. Watchdog Only
 
-Use the --config-url flag to specify the remote configuration URL. For example:
+#### Local Config
+
+```bash
+go run ./cmd/watchdog/ --config-file path/to/config.json
+```
+
+#### Remote Config (with auth)
 
 ```bash
 go run ./cmd/watchdog/ \
   --config-url http://example.com/config.json
 ```
 
-## **Running with Docker**
+See [Endpoints](#endpoints) to access metrics and health checks.
 
-You can also run the application using Docker. Here’s how:
+### 3. Docker (single container)
 
-### 1. **Build the Docker Image**
-
-First, build the Docker image for the application. Navigate to the root of the project directory and run:
+#### Build image
 
 ```bash
 docker build -t watchdog .
 ```
 
-### 2. **Run the Docker Container**
-
-#### 2.1 **Using Local Config**
+#### Run with local config
 
 ```bash
 docker run -d \
   --name watchdog \
-  -e CONFIG_AUTH_USER=admin \
-  -e CONFIG_AUTH_PASS=password \
   -v ./config.json:/app/config.json \
   -p 4000:4000 \
   watchdog \
   --config-file /app/config.json
 ```
 
-#### 2.2 **Using Remote Config**
+#### Run with remote config
 
 ```bash
 docker run -d \
@@ -150,64 +179,50 @@ docker run -d \
   --config-url http://example.com/config.json
 ```
 
-## **Running with Docker Compose**
-
-You can run the Watchdog service along with Prometheus and Grafana for monitoring using Docker Compose.
+See [Endpoints](#endpoints) to access metrics and health checks.
 
-**Important:**
+## Endpoints
 
-> Make sure `config.json` exists before running any Docker commands.  
-> It must be a **file**, not a folder.
+During **development** (using `localhost`):
 
-### **Start all services**
+- Watchdog Metrics: [http://localhost:4000/metrics](http://localhost:4000/metrics)
+- Watchdog Health Check: [http://localhost:4000/v1/healthcheck](http://localhost:4000/v1/healthcheck)
+- Grafana: [http://localhost:3000/login](http://localhost:3000/login) → default user/pass: `admin` / `admin`
+- Prometheus Targets: [http://localhost:9090/targets](http://localhost:9090/targets)
+- Prometheus Query: [http://localhost:9090/query](http://localhost:9090/query)
 
-```bash
-docker compose up -d
-```
+During **production** (replace `<server-ip-or-domain>`):
 
-This will start:
+- Watchdog Metrics: `http://<server-ip-or-domain>:4000/metrics`
+- Watchdog Health Check: `http://<server-ip-or-domain>:4000/v1/healthcheck`
+- Grafana: `http://<server-ip-or-domain>:3000/login`
+- Prometheus Targets: `http://<server-ip-or-domain>:9090/targets`
+- Prometheus Query: `http://<server-ip-or-domain>:9090/query`
 
-- **Watchdog** on port `4000`
-- **Prometheus** on port `9090`
-- **Grafana** on port `3000`
+## Testing
 
-### **Stop all services**
-
-```bash
-docker compose down
-```
-
-### **Restart all services**
-
-```bash
-docker compose restart
-```
-
-Grafana will automatically load the pre-provisioned Go runtime dashboard, and Prometheus will be configured to scrape Watchdog metrics.
-
-# Testing
-
-### To run all unit test cases:
+### Unit Tests
 
 ```bash
 go test ./...
 ```
 
-### To run integration testing
-
-Follow these steps:
+### Integration Tests
 
-1. Open the file `integration_servers.json.template` inside the `internal/integration` package.
-2. Rename it to `integration_servers.json`.
-3. Fill in your OBA server configuration values.
-
-Then run:
+- Copy `integration_config.json.template` → `integration_config.json`
+- Fill in OBA server values
+- Run:
 
 ```bash
-go test -tags=integration ./internal/integration -integration-config ./integration_servers.json
+go test -tags=integration ./internal/integration \
+  -integration-config path/to/integration_config.json
 ```
 
 Note:
 
-- The `integration_servers.json` file is ignored by Git to prevent accidental commits of sensitive data.
-- You can point to any config file by passing its path using the -integration-config flag.
+- ⚠️ the file **must** be named `integration_config.json`
+- It’s Git-ignored for safety
+
+## Contributing
+
+Refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for detailed contribution guidelines.