ocean-cleanup/server/HOTSPOT_CALCULATION_IMPLEMENTATION.md

# Hotspot Calculation System Implementation

## Overview

The hotspot calculation system identifies and tracks areas with high concentrations of litter reports. It automatically calculates hotspots every 5 minutes using MySQL spatial queries and stores the results for efficient retrieval.

## Requirements Addressed

- **Requirement 3.2**: Identify areas with at least 5 reports within a 500-meter radius over the past 30 days
- **Requirement 3.3**: Rank Litter Hotspots by severity score calculated from report quantity and frequency
- **Requirement 3.4**: Update Litter Hotspot calculations within 5 minutes of new report submissions
- **Requirement 3.5**: Display Litter Hotspot boundaries on the Map Interface with color-coded severity indicators

## Architecture

### Components

1. **Spatial Utilities** (`server/src/utils/spatial.ts`)
   - `calculateHotspots()`: Performs MySQL spatial clustering using ST_GeoHash
   - `updateHotspotsTable()`: Clears old hotspots and inserts new calculations

2. **Analytics Service** (`server/src/services/analyticsService.ts`)
   - `getHotspots()`: Retrieves current hotspots from database
   - `calculateAndUpdateHotspots()`: Triggers hotspot calculation
   - `getHotspotDetails(id)`: Gets detailed information about a specific hotspot

3. **Analytics Controller** (`server/src/controllers/analyticsController.ts`)
   - `getHotspots()`: GET endpoint handler
   - `getHotspotDetails()`: GET endpoint handler for specific hotspot
   - `calculateHotspots()`: POST endpoint handler for manual calculation

4. **Scheduled Jobs** (`server/src/services/scheduledJobs.ts`)
   - Runs hotspot calculation every 5 minutes using node-cron
   - Initialized on server startup

## API Endpoints

### GET /api/analytics/hotspots

Retrieves all current hotspots (calculated within the last hour).

**Response:**
```json
{
  "success": true,
  "data": [
    {
      "id": "uuid",
      "latitude": 53.3498,
      "longitude": -6.2603,
      "radius": 500,
      "reportCount": 12,
      "severityScore": 28.5,
      "lastReportDate": "2025-11-17T00:00:00.000Z",
      "calculatedAt": "2025-11-17T00:15:00.000Z"
    }
  ]
}
```

### GET /api/analytics/hotspots/:id

Retrieves detailed information about a specific hotspot, including all reports within its radius.

**Response:**
```json
{
  "success": true,
  "data": {
    "hotspot": {
      "id": "uuid",
      "latitude": 53.3498,
      "longitude": -6.2603,
      "radius": 500,
      "reportCount": 12,
      "severityScore": 28.5,
      "lastReportDate": "2025-11-17T00:00:00.000Z",
      "calculatedAt": "2025-11-17T00:15:00.000Z"
    },
    "reports": [
      {
        "id": "uuid",
        "latitude": 53.3500,
        "longitude": -6.2600,
        "litterType": "plastic",
        "quantity": "significant",
        "createdAt": "2025-11-16T12:00:00.000Z",
        "distance": 45.2
      }
    ]
  }
}
```

### POST /api/analytics/hotspots/calculate

Manually triggers hotspot calculation (useful for testing or immediate updates).

**Response:**
```json
{
  "success": true,
  "data": {
    "success": true,
    "hotspotsCalculated": 3,
    "timestamp": "2025-11-17T00:15:00.000Z"
  }
}
```

## Calculation Algorithm

### Step 1: Spatial Clustering

The system uses MySQL's `ST_GeoHash()` function to group nearby reports:

```sql
SELECT 
  AVG(latitude) as center_lat,
  AVG(longitude) as center_lng,
  COUNT(*) as report_count,
  AVG(severity_value) as avg_severity,
  MAX(created_at) as last_report_date
FROM reports
WHERE created_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)
GROUP BY ST_GeoHash(location, 7)
HAVING COUNT(*) >= 5
```

### Step 2: Severity Score Calculation

Severity score is calculated as:
```
severity_score = report_count × avg_severity
```

Where severity values are:
- minimal: 1
- moderate: 2
- significant: 3
- severe: 4

### Step 3: Storage

Hotspots are stored in the `hotspots` table with:
- Geographic center point (POINT spatial type)
- 500m radius
- Report count
- Severity score
- Last report date
- Calculation timestamp

### Step 4: Cleanup

Old hotspots (calculated more than 1 hour ago) are automatically removed before new calculations.

## Scheduled Job

The hotspot calculation runs automatically every 5 minutes:

```typescript
cron.schedule('*/5 * * * *', async () => {
  await analyticsService.calculateAndUpdateHotspots();
});
```

This ensures hotspots are updated within 5 minutes of new report submissions (Requirement 3.4).

## Database Schema

```sql
CREATE TABLE hotspots (
    id VARCHAR(36) PRIMARY KEY DEFAULT (UUID()),
    location POINT NOT NULL SRID 4326,
    latitude DECIMAL(10, 8) NOT NULL,
    longitude DECIMAL(11, 8) NOT NULL,
    radius DECIMAL(10, 2) NOT NULL COMMENT 'Radius in meters',
    
    report_count INT NOT NULL,
    severity_score DECIMAL(5, 2) NOT NULL,
    last_report_date TIMESTAMP NOT NULL,
    
    calculated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    
    SPATIAL INDEX idx_location (location),
    INDEX idx_severity_score (severity_score)
) ENGINE=InnoDB;
```

## Testing

### Unit Tests

Run the hotspot calculation test:
```bash
cd server
npx tsx src/scripts/testHotspots.ts
```

### API Endpoint Tests

1. Start the server:
```bash
cd server
npm run dev
```

2. Run endpoint tests:
```bash
npx tsx src/scripts/testHotspotEndpoints.ts
```

### Manual Testing

```bash
# Get all hotspots
curl http://localhost:3005/api/analytics/hotspots

# Trigger manual calculation
curl -X POST http://localhost:3005/api/analytics/hotspots/calculate

# Get specific hotspot details
curl http://localhost:3005/api/analytics/hotspots/{hotspot-id}
```

## Frontend Integration

To display hotspots on the map:

```typescript
import { useQuery } from '@tanstack/react-query';

function MapView() {
  const { data: hotspots } = useQuery({
    queryKey: ['hotspots'],
    queryFn: async () => {
      const response = await fetch('/api/analytics/hotspots');
      return response.json();
    },
    refetchInterval: 5 * 60 * 1000, // Refetch every 5 minutes
  });

  // Render hotspots as circles on the map
  hotspots?.data.forEach(hotspot => {
    L.circle([hotspot.latitude, hotspot.longitude], {
      radius: hotspot.radius,
      color: getSeverityColor(hotspot.severityScore),
      fillOpacity: 0.3,
    }).addTo(map);
  });
}

function getSeverityColor(score: number): string {
  if (score >= 50) return '#dc2626'; // red - critical
  if (score >= 30) return '#ea580c'; // orange - high
  if (score >= 15) return '#f59e0b'; // yellow - medium
  return '#84cc16'; // green - low
}
```

## Performance Considerations

1. **Spatial Indexes**: The `hotspots` table uses spatial indexes for efficient location queries
2. **Caching**: Hotspots are cached for 1 hour to reduce calculation frequency
3. **Scheduled Updates**: 5-minute intervals balance freshness with server load
4. **Query Optimization**: Uses MySQL spatial functions for efficient clustering

## Monitoring

The scheduled job logs its activity:

```
[Scheduled Jobs] Initializing scheduled jobs...
[Scheduled Jobs] ✓ Hotspot calculation job scheduled (every 5 minutes)
[Scheduled Jobs] Running hotspot calculation...
[Hotspot Calculation] Starting hotspot calculation...
[Hotspot Calculation] Successfully calculated 3 hotspots
[Scheduled Jobs] Hotspot calculation completed: 3 hotspots calculated
```

## Future Enhancements

1. **Historical Tracking**: Store hotspot history for trend analysis
2. **Notification System**: Alert users when new hotspots appear in their areas of interest
3. **Predictive Analysis**: Use ML to predict future hotspot locations
4. **Custom Thresholds**: Allow admins to configure minimum report count and radius
5. **Hotspot Decay**: Reduce severity scores for hotspots with no recent reports

## Troubleshooting

### No Hotspots Calculated

- Verify there are at least 5 reports within 500m in the last 30 days
- Check that reports have valid spatial data
- Review logs for calculation errors

### Scheduled Job Not Running

- Verify node-cron is installed: `npm list node-cron`
- Check server logs for initialization messages
- Ensure server is running continuously

### Spatial Query Errors

- Verify MySQL version is 8.0+ with spatial support
- Check that POINT columns use SRID 4326
- Ensure spatial indexes are created

## Related Files

- `server/src/services/analyticsService.ts` - Hotspot business logic
- `server/src/controllers/analyticsController.ts` - API endpoints
- `server/src/routes/analyticsRoutes.ts` - Route definitions
- `server/src/services/scheduledJobs.ts` - Cron job configuration
- `server/src/utils/spatial.ts` - Spatial utility functions
- `server/prisma/schema.prisma` - Database schema