feat: implement hotspot calculation system with scheduled jobs
- Add hotspot calculation methods to analytics service - Implement MySQL spatial queries for clustering (5+ reports within 500m) - Calculate severity scores based on report count and frequency - Create GET /api/analytics/hotspots endpoint - Create GET /api/analytics/hotspots/:id endpoint for details - Create POST /api/analytics/hotspots/calculate for manual trigger - Implement scheduled job with node-cron (runs every 5 minutes) - Add comprehensive test scripts for hotspot functionality - Create detailed implementation documentation Requirements addressed: - 3.2: Identify areas with 5+ reports within 500m radius - 3.3: Rank hotspots by severity score - 3.4: Update calculations within 5 minutes - 3.5: Provide data for map visualization with severity indicators
This commit is contained in:
316
server/HOTSPOT_CALCULATION_IMPLEMENTATION.md
Normal file
316
server/HOTSPOT_CALCULATION_IMPLEMENTATION.md
Normal file
@@ -0,0 +1,316 @@
|
||||
# Hotspot Calculation System Implementation
|
||||
|
||||
## Overview
|
||||
|
||||
The hotspot calculation system identifies and tracks areas with high concentrations of litter reports. It automatically calculates hotspots every 5 minutes using MySQL spatial queries and stores the results for efficient retrieval.
|
||||
|
||||
## Requirements Addressed
|
||||
|
||||
- **Requirement 3.2**: Identify areas with at least 5 reports within a 500-meter radius over the past 30 days
|
||||
- **Requirement 3.3**: Rank Litter Hotspots by severity score calculated from report quantity and frequency
|
||||
- **Requirement 3.4**: Update Litter Hotspot calculations within 5 minutes of new report submissions
|
||||
- **Requirement 3.5**: Display Litter Hotspot boundaries on the Map Interface with color-coded severity indicators
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
1. **Spatial Utilities** (`server/src/utils/spatial.ts`)
|
||||
- `calculateHotspots()`: Performs MySQL spatial clustering using ST_GeoHash
|
||||
- `updateHotspotsTable()`: Clears old hotspots and inserts new calculations
|
||||
|
||||
2. **Analytics Service** (`server/src/services/analyticsService.ts`)
|
||||
- `getHotspots()`: Retrieves current hotspots from database
|
||||
- `calculateAndUpdateHotspots()`: Triggers hotspot calculation
|
||||
- `getHotspotDetails(id)`: Gets detailed information about a specific hotspot
|
||||
|
||||
3. **Analytics Controller** (`server/src/controllers/analyticsController.ts`)
|
||||
- `getHotspots()`: GET endpoint handler
|
||||
- `getHotspotDetails()`: GET endpoint handler for specific hotspot
|
||||
- `calculateHotspots()`: POST endpoint handler for manual calculation
|
||||
|
||||
4. **Scheduled Jobs** (`server/src/services/scheduledJobs.ts`)
|
||||
- Runs hotspot calculation every 5 minutes using node-cron
|
||||
- Initialized on server startup
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### GET /api/analytics/hotspots
|
||||
|
||||
Retrieves all current hotspots (calculated within the last hour).
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"latitude": 53.3498,
|
||||
"longitude": -6.2603,
|
||||
"radius": 500,
|
||||
"reportCount": 12,
|
||||
"severityScore": 28.5,
|
||||
"lastReportDate": "2025-11-17T00:00:00.000Z",
|
||||
"calculatedAt": "2025-11-17T00:15:00.000Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/analytics/hotspots/:id
|
||||
|
||||
Retrieves detailed information about a specific hotspot, including all reports within its radius.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"hotspot": {
|
||||
"id": "uuid",
|
||||
"latitude": 53.3498,
|
||||
"longitude": -6.2603,
|
||||
"radius": 500,
|
||||
"reportCount": 12,
|
||||
"severityScore": 28.5,
|
||||
"lastReportDate": "2025-11-17T00:00:00.000Z",
|
||||
"calculatedAt": "2025-11-17T00:15:00.000Z"
|
||||
},
|
||||
"reports": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"latitude": 53.3500,
|
||||
"longitude": -6.2600,
|
||||
"litterType": "plastic",
|
||||
"quantity": "significant",
|
||||
"createdAt": "2025-11-16T12:00:00.000Z",
|
||||
"distance": 45.2
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/analytics/hotspots/calculate
|
||||
|
||||
Manually triggers hotspot calculation (useful for testing or immediate updates).
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"success": true,
|
||||
"hotspotsCalculated": 3,
|
||||
"timestamp": "2025-11-17T00:15:00.000Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Calculation Algorithm
|
||||
|
||||
### Step 1: Spatial Clustering
|
||||
|
||||
The system uses MySQL's `ST_GeoHash()` function to group nearby reports:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
AVG(latitude) as center_lat,
|
||||
AVG(longitude) as center_lng,
|
||||
COUNT(*) as report_count,
|
||||
AVG(severity_value) as avg_severity,
|
||||
MAX(created_at) as last_report_date
|
||||
FROM reports
|
||||
WHERE created_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)
|
||||
GROUP BY ST_GeoHash(location, 7)
|
||||
HAVING COUNT(*) >= 5
|
||||
```
|
||||
|
||||
### Step 2: Severity Score Calculation
|
||||
|
||||
Severity score is calculated as:
|
||||
```
|
||||
severity_score = report_count × avg_severity
|
||||
```
|
||||
|
||||
Where severity values are:
|
||||
- minimal: 1
|
||||
- moderate: 2
|
||||
- significant: 3
|
||||
- severe: 4
|
||||
|
||||
### Step 3: Storage
|
||||
|
||||
Hotspots are stored in the `hotspots` table with:
|
||||
- Geographic center point (POINT spatial type)
|
||||
- 500m radius
|
||||
- Report count
|
||||
- Severity score
|
||||
- Last report date
|
||||
- Calculation timestamp
|
||||
|
||||
### Step 4: Cleanup
|
||||
|
||||
Old hotspots (calculated more than 1 hour ago) are automatically removed before new calculations.
|
||||
|
||||
## Scheduled Job
|
||||
|
||||
The hotspot calculation runs automatically every 5 minutes:
|
||||
|
||||
```typescript
|
||||
cron.schedule('*/5 * * * *', async () => {
|
||||
await analyticsService.calculateAndUpdateHotspots();
|
||||
});
|
||||
```
|
||||
|
||||
This ensures hotspots are updated within 5 minutes of new report submissions (Requirement 3.4).
|
||||
|
||||
## Database Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE hotspots (
|
||||
id VARCHAR(36) PRIMARY KEY DEFAULT (UUID()),
|
||||
location POINT NOT NULL SRID 4326,
|
||||
latitude DECIMAL(10, 8) NOT NULL,
|
||||
longitude DECIMAL(11, 8) NOT NULL,
|
||||
radius DECIMAL(10, 2) NOT NULL COMMENT 'Radius in meters',
|
||||
|
||||
report_count INT NOT NULL,
|
||||
severity_score DECIMAL(5, 2) NOT NULL,
|
||||
last_report_date TIMESTAMP NOT NULL,
|
||||
|
||||
calculated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
SPATIAL INDEX idx_location (location),
|
||||
INDEX idx_severity_score (severity_score)
|
||||
) ENGINE=InnoDB;
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
Run the hotspot calculation test:
|
||||
```bash
|
||||
cd server
|
||||
npx tsx src/scripts/testHotspots.ts
|
||||
```
|
||||
|
||||
### API Endpoint Tests
|
||||
|
||||
1. Start the server:
|
||||
```bash
|
||||
cd server
|
||||
npm run dev
|
||||
```
|
||||
|
||||
2. Run endpoint tests:
|
||||
```bash
|
||||
npx tsx src/scripts/testHotspotEndpoints.ts
|
||||
```
|
||||
|
||||
### Manual Testing
|
||||
|
||||
```bash
|
||||
# Get all hotspots
|
||||
curl http://localhost:3005/api/analytics/hotspots
|
||||
|
||||
# Trigger manual calculation
|
||||
curl -X POST http://localhost:3005/api/analytics/hotspots/calculate
|
||||
|
||||
# Get specific hotspot details
|
||||
curl http://localhost:3005/api/analytics/hotspots/{hotspot-id}
|
||||
```
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
To display hotspots on the map:
|
||||
|
||||
```typescript
|
||||
import { useQuery } from '@tanstack/react-query';
|
||||
|
||||
function MapView() {
|
||||
const { data: hotspots } = useQuery({
|
||||
queryKey: ['hotspots'],
|
||||
queryFn: async () => {
|
||||
const response = await fetch('/api/analytics/hotspots');
|
||||
return response.json();
|
||||
},
|
||||
refetchInterval: 5 * 60 * 1000, // Refetch every 5 minutes
|
||||
});
|
||||
|
||||
// Render hotspots as circles on the map
|
||||
hotspots?.data.forEach(hotspot => {
|
||||
L.circle([hotspot.latitude, hotspot.longitude], {
|
||||
radius: hotspot.radius,
|
||||
color: getSeverityColor(hotspot.severityScore),
|
||||
fillOpacity: 0.3,
|
||||
}).addTo(map);
|
||||
});
|
||||
}
|
||||
|
||||
function getSeverityColor(score: number): string {
|
||||
if (score >= 50) return '#dc2626'; // red - critical
|
||||
if (score >= 30) return '#ea580c'; // orange - high
|
||||
if (score >= 15) return '#f59e0b'; // yellow - medium
|
||||
return '#84cc16'; // green - low
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Spatial Indexes**: The `hotspots` table uses spatial indexes for efficient location queries
|
||||
2. **Caching**: Hotspots are cached for 1 hour to reduce calculation frequency
|
||||
3. **Scheduled Updates**: 5-minute intervals balance freshness with server load
|
||||
4. **Query Optimization**: Uses MySQL spatial functions for efficient clustering
|
||||
|
||||
## Monitoring
|
||||
|
||||
The scheduled job logs its activity:
|
||||
|
||||
```
|
||||
[Scheduled Jobs] Initializing scheduled jobs...
|
||||
[Scheduled Jobs] ✓ Hotspot calculation job scheduled (every 5 minutes)
|
||||
[Scheduled Jobs] Running hotspot calculation...
|
||||
[Hotspot Calculation] Starting hotspot calculation...
|
||||
[Hotspot Calculation] Successfully calculated 3 hotspots
|
||||
[Scheduled Jobs] Hotspot calculation completed: 3 hotspots calculated
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Historical Tracking**: Store hotspot history for trend analysis
|
||||
2. **Notification System**: Alert users when new hotspots appear in their areas of interest
|
||||
3. **Predictive Analysis**: Use ML to predict future hotspot locations
|
||||
4. **Custom Thresholds**: Allow admins to configure minimum report count and radius
|
||||
5. **Hotspot Decay**: Reduce severity scores for hotspots with no recent reports
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No Hotspots Calculated
|
||||
|
||||
- Verify there are at least 5 reports within 500m in the last 30 days
|
||||
- Check that reports have valid spatial data
|
||||
- Review logs for calculation errors
|
||||
|
||||
### Scheduled Job Not Running
|
||||
|
||||
- Verify node-cron is installed: `npm list node-cron`
|
||||
- Check server logs for initialization messages
|
||||
- Ensure server is running continuously
|
||||
|
||||
### Spatial Query Errors
|
||||
|
||||
- Verify MySQL version is 8.0+ with spatial support
|
||||
- Check that POINT columns use SRID 4326
|
||||
- Ensure spatial indexes are created
|
||||
|
||||
## Related Files
|
||||
|
||||
- `server/src/services/analyticsService.ts` - Hotspot business logic
|
||||
- `server/src/controllers/analyticsController.ts` - API endpoints
|
||||
- `server/src/routes/analyticsRoutes.ts` - Route definitions
|
||||
- `server/src/services/scheduledJobs.ts` - Cron job configuration
|
||||
- `server/src/utils/spatial.ts` - Spatial utility functions
|
||||
- `server/prisma/schema.prisma` - Database schema
|
||||
@@ -196,6 +196,93 @@ export class AnalyticsController {
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/analytics/hotspots
|
||||
* Get litter hotspot data
|
||||
* Requirement 3.2, 3.3, 3.5: Display hotspots with severity indicators
|
||||
*/
|
||||
async getHotspots(req: Request, res: Response) {
|
||||
try {
|
||||
const hotspots = await analyticsService.getHotspots();
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: hotspots,
|
||||
});
|
||||
} catch (error: any) {
|
||||
console.error('Error fetching hotspots:', error);
|
||||
res.status(500).json({
|
||||
error: {
|
||||
code: 'INTERNAL_SERVER_ERROR',
|
||||
message: error.message || 'Failed to fetch hotspots',
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/analytics/hotspots/:id
|
||||
* Get detailed information about a specific hotspot
|
||||
*/
|
||||
async getHotspotDetails(req: Request, res: Response) {
|
||||
try {
|
||||
const { id } = req.params;
|
||||
|
||||
const hotspotDetails = await analyticsService.getHotspotDetails(id);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: hotspotDetails,
|
||||
});
|
||||
} catch (error: any) {
|
||||
console.error('Error fetching hotspot details:', error);
|
||||
|
||||
if (error.message === 'Hotspot not found') {
|
||||
return res.status(404).json({
|
||||
error: {
|
||||
code: 'RESOURCE_NOT_FOUND',
|
||||
message: 'Hotspot not found',
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
res.status(500).json({
|
||||
error: {
|
||||
code: 'INTERNAL_SERVER_ERROR',
|
||||
message: error.message || 'Failed to fetch hotspot details',
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/analytics/hotspots/calculate
|
||||
* Manually trigger hotspot calculation
|
||||
* Requirement 3.4: Update Litter Hotspot calculations
|
||||
*/
|
||||
async calculateHotspots(req: Request, res: Response) {
|
||||
try {
|
||||
const result = await analyticsService.calculateAndUpdateHotspots();
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: result,
|
||||
});
|
||||
} catch (error: any) {
|
||||
console.error('Error calculating hotspots:', error);
|
||||
res.status(500).json({
|
||||
error: {
|
||||
code: 'INTERNAL_SERVER_ERROR',
|
||||
message: error.message || 'Failed to calculate hotspots',
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const analyticsController = new AnalyticsController();
|
||||
|
||||
@@ -8,6 +8,7 @@ import eventRoutes from './routes/eventRoutes';
|
||||
import userRoutes from './routes/userRoutes';
|
||||
import externalDataRoutes from './routes/externalDataRoutes';
|
||||
import analyticsRoutes from './routes/analyticsRoutes';
|
||||
import { initializeScheduledJobs } from './services/scheduledJobs';
|
||||
|
||||
// Load environment variables
|
||||
dotenv.config();
|
||||
@@ -77,6 +78,10 @@ app.listen(PORT, () => {
|
||||
console.log(`🚀 Server running on port ${PORT}`);
|
||||
console.log(`📍 Environment: ${process.env.NODE_ENV || 'development'}`);
|
||||
console.log(`🔗 API: http://localhost:${PORT}/api`);
|
||||
|
||||
// Initialize scheduled jobs
|
||||
console.log('⏰ Initializing scheduled jobs...');
|
||||
initializeScheduledJobs();
|
||||
});
|
||||
|
||||
export default app;
|
||||
|
||||
@@ -10,4 +10,9 @@ router.get('/comparison', analyticsController.getComparison.bind(analyticsContro
|
||||
router.get('/export', analyticsController.exportCSV.bind(analyticsController));
|
||||
router.get('/aggregated', analyticsController.getAggregatedData.bind(analyticsController));
|
||||
|
||||
// Hotspot endpoints
|
||||
router.get('/hotspots', analyticsController.getHotspots.bind(analyticsController));
|
||||
router.get('/hotspots/:id', analyticsController.getHotspotDetails.bind(analyticsController));
|
||||
router.post('/hotspots/calculate', analyticsController.calculateHotspots.bind(analyticsController));
|
||||
|
||||
export default router;
|
||||
|
||||
104
server/src/scripts/testHotspotEndpoints.ts
Normal file
104
server/src/scripts/testHotspotEndpoints.ts
Normal file
@@ -0,0 +1,104 @@
|
||||
/**
|
||||
* Test script for hotspot API endpoints
|
||||
* Tests GET /api/analytics/hotspots and POST /api/analytics/hotspots/calculate
|
||||
*/
|
||||
|
||||
import axios from 'axios';
|
||||
|
||||
const API_BASE_URL = process.env.API_BASE_URL || 'http://localhost:3005/api';
|
||||
|
||||
async function testHotspotEndpoints() {
|
||||
console.log('=== Testing Hotspot API Endpoints ===\n');
|
||||
|
||||
try {
|
||||
// Test 1: Manually trigger hotspot calculation
|
||||
console.log('Test 1: POST /api/analytics/hotspots/calculate');
|
||||
console.log('----------------------------------------');
|
||||
try {
|
||||
const calculateResponse = await axios.post(`${API_BASE_URL}/analytics/hotspots/calculate`);
|
||||
console.log('✓ Status:', calculateResponse.status);
|
||||
console.log('✓ Response:', JSON.stringify(calculateResponse.data, null, 2));
|
||||
} catch (error: any) {
|
||||
if (error.response) {
|
||||
console.log('✓ Status:', error.response.status);
|
||||
console.log('✓ Response:', JSON.stringify(error.response.data, null, 2));
|
||||
} else {
|
||||
console.error('❌ Error:', error.message);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Test 2: Get all hotspots
|
||||
console.log('Test 2: GET /api/analytics/hotspots');
|
||||
console.log('----------------------------------------');
|
||||
try {
|
||||
const hotspotsResponse = await axios.get(`${API_BASE_URL}/analytics/hotspots`);
|
||||
console.log('✓ Status:', hotspotsResponse.status);
|
||||
console.log('✓ Number of hotspots:', hotspotsResponse.data.data.length);
|
||||
|
||||
if (hotspotsResponse.data.data.length > 0) {
|
||||
console.log('✓ Sample hotspot:', JSON.stringify(hotspotsResponse.data.data[0], null, 2));
|
||||
} else {
|
||||
console.log('✓ No hotspots found (expected if < 5 reports within 500m in last 30 days)');
|
||||
}
|
||||
} catch (error: any) {
|
||||
if (error.response) {
|
||||
console.log('❌ Status:', error.response.status);
|
||||
console.log('❌ Response:', JSON.stringify(error.response.data, null, 2));
|
||||
} else {
|
||||
console.error('❌ Error:', error.message);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Test 3: Get hotspot details (if any exist)
|
||||
console.log('Test 3: GET /api/analytics/hotspots/:id');
|
||||
console.log('----------------------------------------');
|
||||
try {
|
||||
const hotspotsResponse = await axios.get(`${API_BASE_URL}/analytics/hotspots`);
|
||||
|
||||
if (hotspotsResponse.data.data.length > 0) {
|
||||
const firstHotspotId = hotspotsResponse.data.data[0].id;
|
||||
const detailsResponse = await axios.get(`${API_BASE_URL}/analytics/hotspots/${firstHotspotId}`);
|
||||
console.log('✓ Status:', detailsResponse.status);
|
||||
console.log('✓ Hotspot details:', JSON.stringify(detailsResponse.data.data.hotspot, null, 2));
|
||||
console.log('✓ Number of reports in hotspot:', detailsResponse.data.data.reports.length);
|
||||
} else {
|
||||
console.log('⊘ Skipping test - no hotspots available');
|
||||
}
|
||||
} catch (error: any) {
|
||||
if (error.response) {
|
||||
console.log('❌ Status:', error.response.status);
|
||||
console.log('❌ Response:', JSON.stringify(error.response.data, null, 2));
|
||||
} else {
|
||||
console.error('❌ Error:', error.message);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Test 4: Test error handling - invalid hotspot ID
|
||||
console.log('Test 4: GET /api/analytics/hotspots/:id (invalid ID)');
|
||||
console.log('----------------------------------------');
|
||||
try {
|
||||
await axios.get(`${API_BASE_URL}/analytics/hotspots/invalid-id-12345`);
|
||||
console.log('❌ Should have returned 404 error');
|
||||
} catch (error: any) {
|
||||
if (error.response && error.response.status === 404) {
|
||||
console.log('✓ Status:', error.response.status);
|
||||
console.log('✓ Correctly returned 404 for invalid ID');
|
||||
console.log('✓ Error response:', JSON.stringify(error.response.data, null, 2));
|
||||
} else {
|
||||
console.error('❌ Unexpected error:', error.message);
|
||||
}
|
||||
}
|
||||
console.log();
|
||||
|
||||
console.log('=== All Endpoint Tests Completed ===');
|
||||
} catch (error: any) {
|
||||
console.error('❌ Fatal error during testing:', error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Run tests
|
||||
console.log('Note: Make sure the server is running on port 3005 before running this test.\n');
|
||||
testHotspotEndpoints();
|
||||
98
server/src/scripts/testHotspots.ts
Normal file
98
server/src/scripts/testHotspots.ts
Normal file
@@ -0,0 +1,98 @@
|
||||
/**
|
||||
* Test script for hotspot calculation system
|
||||
* Tests the hotspot calculation, storage, and retrieval functionality
|
||||
*/
|
||||
|
||||
import { analyticsService } from '../services/analyticsService';
|
||||
import prisma from '../config/database';
|
||||
|
||||
async function testHotspots() {
|
||||
console.log('=== Testing Hotspot Calculation System ===\n');
|
||||
|
||||
try {
|
||||
// Test 1: Calculate and update hotspots
|
||||
console.log('Test 1: Calculate and update hotspots');
|
||||
console.log('----------------------------------------');
|
||||
const calculateResult = await analyticsService.calculateAndUpdateHotspots();
|
||||
console.log('✓ Hotspot calculation result:', calculateResult);
|
||||
console.log();
|
||||
|
||||
// Test 2: Retrieve hotspots
|
||||
console.log('Test 2: Retrieve hotspots from database');
|
||||
console.log('----------------------------------------');
|
||||
const hotspots = await analyticsService.getHotspots();
|
||||
console.log(`✓ Found ${hotspots.length} hotspots`);
|
||||
|
||||
if (hotspots.length > 0) {
|
||||
console.log('\nTop 3 hotspots by severity:');
|
||||
hotspots.slice(0, 3).forEach((hotspot, index) => {
|
||||
console.log(` ${index + 1}. Hotspot at (${hotspot.latitude}, ${hotspot.longitude})`);
|
||||
console.log(` - Report Count: ${hotspot.reportCount}`);
|
||||
console.log(` - Severity Score: ${hotspot.severityScore}`);
|
||||
console.log(` - Radius: ${hotspot.radius}m`);
|
||||
console.log(` - Last Report: ${hotspot.lastReportDate}`);
|
||||
});
|
||||
} else {
|
||||
console.log(' No hotspots found. This is expected if there are fewer than 5 reports within 500m in the last 30 days.');
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Test 3: Get hotspot details (if any exist)
|
||||
if (hotspots.length > 0) {
|
||||
console.log('Test 3: Get detailed hotspot information');
|
||||
console.log('----------------------------------------');
|
||||
const firstHotspot = hotspots[0];
|
||||
const details = await analyticsService.getHotspotDetails(firstHotspot.id);
|
||||
console.log(`✓ Hotspot details for ${firstHotspot.id}:`);
|
||||
console.log(` - Location: (${details.hotspot.latitude}, ${details.hotspot.longitude})`);
|
||||
console.log(` - Reports in area: ${details.reports.length}`);
|
||||
|
||||
if (details.reports.length > 0) {
|
||||
console.log('\n Recent reports in this hotspot:');
|
||||
details.reports.slice(0, 3).forEach((report, index) => {
|
||||
console.log(` ${index + 1}. ${report.litterType} (${report.quantity}) - ${report.distance.toFixed(0)}m away`);
|
||||
});
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
|
||||
// Test 4: Check report distribution for hotspot eligibility
|
||||
console.log('Test 4: Check report distribution');
|
||||
console.log('----------------------------------------');
|
||||
const recentReports = await prisma.report.count({
|
||||
where: {
|
||||
createdAt: {
|
||||
gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000), // Last 30 days
|
||||
},
|
||||
},
|
||||
});
|
||||
console.log(`✓ Total reports in last 30 days: ${recentReports}`);
|
||||
console.log(` Note: Hotspots require 5+ reports within 500m radius`);
|
||||
console.log();
|
||||
|
||||
// Test 5: Verify hotspot table structure
|
||||
console.log('Test 5: Verify hotspot table structure');
|
||||
console.log('----------------------------------------');
|
||||
const allHotspots = await prisma.hotspot.findMany({
|
||||
take: 1,
|
||||
});
|
||||
|
||||
if (allHotspots.length > 0) {
|
||||
console.log('✓ Hotspot table structure verified');
|
||||
console.log(' Sample hotspot fields:', Object.keys(allHotspots[0]));
|
||||
} else {
|
||||
console.log('✓ Hotspot table exists but is empty');
|
||||
}
|
||||
console.log();
|
||||
|
||||
console.log('=== All Tests Completed Successfully ===');
|
||||
} catch (error: any) {
|
||||
console.error('❌ Error during testing:', error.message);
|
||||
console.error(error);
|
||||
} finally {
|
||||
await prisma.$disconnect();
|
||||
}
|
||||
}
|
||||
|
||||
// Run tests
|
||||
testHotspots();
|
||||
@@ -1,5 +1,6 @@
|
||||
import prisma from '../config/database';
|
||||
import { Parser } from 'json2csv';
|
||||
import { calculateHotspots, updateHotspotsTable } from '../utils/spatial';
|
||||
|
||||
interface DateRange {
|
||||
startDate: Date;
|
||||
@@ -525,6 +526,129 @@ export class AnalyticsService {
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get hotspot data from the database
|
||||
* Requirement 3.2: Identify areas with at least 5 reports within a 500-meter radius over the past 30 days
|
||||
* Requirement 3.3: Rank Litter Hotspots by severity score calculated from report quantity and frequency
|
||||
* Requirement 3.5: Display Litter Hotspot boundaries on the Map Interface with color-coded severity indicators
|
||||
*/
|
||||
async getHotspots() {
|
||||
const hotspots = await prisma.hotspot.findMany({
|
||||
where: {
|
||||
calculatedAt: {
|
||||
gte: new Date(Date.now() - 60 * 60 * 1000), // Last hour
|
||||
},
|
||||
},
|
||||
orderBy: {
|
||||
severityScore: 'desc',
|
||||
},
|
||||
});
|
||||
|
||||
return hotspots.map(hotspot => ({
|
||||
id: hotspot.id,
|
||||
latitude: Number(hotspot.latitude),
|
||||
longitude: Number(hotspot.longitude),
|
||||
radius: Number(hotspot.radius),
|
||||
reportCount: hotspot.reportCount,
|
||||
severityScore: Number(hotspot.severityScore),
|
||||
lastReportDate: hotspot.lastReportDate.toISOString(),
|
||||
calculatedAt: hotspot.calculatedAt?.toISOString(),
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate and update hotspots
|
||||
* This method performs the actual hotspot calculation and stores results
|
||||
* Requirement 3.2: Identify areas with at least 5 reports within a 500-meter radius
|
||||
* Requirement 3.3: Calculate severity scores based on report count and frequency
|
||||
* Requirement 3.4: Update Litter Hotspot calculations within 5 minutes of new report submissions
|
||||
*/
|
||||
async calculateAndUpdateHotspots() {
|
||||
try {
|
||||
console.log('[Hotspot Calculation] Starting hotspot calculation...');
|
||||
|
||||
// Use the spatial utility function to update hotspots
|
||||
const hotspotsCount = await updateHotspotsTable();
|
||||
|
||||
console.log(`[Hotspot Calculation] Successfully calculated ${hotspotsCount} hotspots`);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
hotspotsCalculated: hotspotsCount,
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
} catch (error: any) {
|
||||
console.error('[Hotspot Calculation] Error calculating hotspots:', error);
|
||||
throw new Error(`Failed to calculate hotspots: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get detailed hotspot information including nearby reports
|
||||
* @param hotspotId - The ID of the hotspot
|
||||
*/
|
||||
async getHotspotDetails(hotspotId: string) {
|
||||
const hotspot = await prisma.hotspot.findUnique({
|
||||
where: { id: hotspotId },
|
||||
});
|
||||
|
||||
if (!hotspot) {
|
||||
throw new Error('Hotspot not found');
|
||||
}
|
||||
|
||||
// Find reports near this hotspot
|
||||
const reports = await prisma.$queryRaw<Array<{
|
||||
id: string;
|
||||
latitude: number;
|
||||
longitude: number;
|
||||
litter_type: string;
|
||||
quantity: string;
|
||||
created_at: Date;
|
||||
distance: number;
|
||||
}>>`
|
||||
SELECT
|
||||
id,
|
||||
latitude,
|
||||
longitude,
|
||||
litter_type,
|
||||
quantity,
|
||||
created_at,
|
||||
ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText(${`POINT(${hotspot.longitude} ${hotspot.latitude})`}, 4326)
|
||||
) as distance
|
||||
FROM reports
|
||||
WHERE ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText(${`POINT(${hotspot.longitude} ${hotspot.latitude})`}, 4326)
|
||||
) <= ${Number(hotspot.radius)}
|
||||
AND created_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)
|
||||
ORDER BY created_at DESC
|
||||
`;
|
||||
|
||||
return {
|
||||
hotspot: {
|
||||
id: hotspot.id,
|
||||
latitude: Number(hotspot.latitude),
|
||||
longitude: Number(hotspot.longitude),
|
||||
radius: Number(hotspot.radius),
|
||||
reportCount: hotspot.reportCount,
|
||||
severityScore: Number(hotspot.severityScore),
|
||||
lastReportDate: hotspot.lastReportDate.toISOString(),
|
||||
calculatedAt: hotspot.calculatedAt?.toISOString(),
|
||||
},
|
||||
reports: reports.map(report => ({
|
||||
id: report.id,
|
||||
latitude: Number(report.latitude),
|
||||
longitude: Number(report.longitude),
|
||||
litterType: report.litter_type,
|
||||
quantity: report.quantity,
|
||||
createdAt: report.created_at.toISOString(),
|
||||
distance: Number(report.distance),
|
||||
})),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export const analyticsService = new AnalyticsService();
|
||||
|
||||
53
server/src/services/scheduledJobs.ts
Normal file
53
server/src/services/scheduledJobs.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
/**
|
||||
* Scheduled Jobs Service
|
||||
* Manages periodic tasks using node-cron
|
||||
*
|
||||
* Requirement 3.4: Update Litter Hotspot calculations within 5 minutes of new report submissions
|
||||
*/
|
||||
|
||||
import cron, { ScheduledTask } from 'node-cron';
|
||||
import { analyticsService } from './analyticsService';
|
||||
|
||||
/**
|
||||
* Initialize all scheduled jobs
|
||||
*/
|
||||
export function initializeScheduledJobs() {
|
||||
console.log('[Scheduled Jobs] Initializing scheduled jobs...');
|
||||
|
||||
// Hotspot calculation job - runs every 5 minutes
|
||||
// Requirement 3.4: Update hotspot calculations within 5 minutes
|
||||
const hotspotJob = cron.schedule('*/5 * * * *', async () => {
|
||||
try {
|
||||
console.log('[Scheduled Jobs] Running hotspot calculation...');
|
||||
const result = await analyticsService.calculateAndUpdateHotspots();
|
||||
console.log(`[Scheduled Jobs] Hotspot calculation completed: ${result.hotspotsCalculated} hotspots calculated`);
|
||||
} catch (error: any) {
|
||||
console.error('[Scheduled Jobs] Error in hotspot calculation job:', error.message);
|
||||
}
|
||||
});
|
||||
|
||||
console.log('[Scheduled Jobs] ✓ Hotspot calculation job scheduled (every 5 minutes)');
|
||||
|
||||
// Return job instances for potential management
|
||||
return {
|
||||
hotspotJob,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop all scheduled jobs
|
||||
* Useful for graceful shutdown
|
||||
*/
|
||||
export function stopScheduledJobs(jobs: { hotspotJob: ScheduledTask }) {
|
||||
console.log('[Scheduled Jobs] Stopping all scheduled jobs...');
|
||||
|
||||
if (jobs.hotspotJob) {
|
||||
jobs.hotspotJob.stop();
|
||||
console.log('[Scheduled Jobs] ✓ Hotspot calculation job stopped');
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
initializeScheduledJobs,
|
||||
stopScheduledJobs,
|
||||
};
|
||||
Reference in New Issue
Block a user