feat: Configure database with Prisma and MySQL spatial support (Task 22)
- Set up MySQL 8.0+ with spatial data support (POINT SRID 4326) - Created Prisma schema with all models (User, Report, Event, etc.) - Added Clerk authentication support (clerk_id field) - Configured spatial indexes on location fields for performance - Created database migrations and applied to MySQL - Built spatial query utilities for geographical operations - Added database connection singleton and health checks - Created comprehensive documentation and test scripts Requirements: 1.1, 2.1, 3.2, 4.1
This commit is contained in:
253
server/prisma/README.md
Normal file
253
server/prisma/README.md
Normal file
@@ -0,0 +1,253 @@
|
||||
# Database Configuration
|
||||
|
||||
## Overview
|
||||
|
||||
This project uses **MySQL 8.0+** with spatial data support for storing geographical information about beach litter reports, cleanup events, and environmental concerns.
|
||||
|
||||
## Database Connection
|
||||
|
||||
**Connection Details:**
|
||||
- Host: 192.168.4.20
|
||||
- Port: 3306
|
||||
- Database: beach_litter_mapping
|
||||
- User: beach_litter_user
|
||||
|
||||
**Environment Variable:**
|
||||
```env
|
||||
DATABASE_URL="mysql://beach_litter_user:BeachClean2024!@192.168.4.20:3306/beach_litter_mapping"
|
||||
```
|
||||
|
||||
## Spatial Data Support
|
||||
|
||||
The database uses MySQL's spatial data types and functions:
|
||||
|
||||
- **POINT type with SRID 4326**: Used for storing GPS coordinates (WGS 84 coordinate system)
|
||||
- **Spatial indexes**: Optimized for location-based queries
|
||||
- Tables with spatial data: `reports`, `events`, `hotspots`
|
||||
|
||||
### Spatial Columns
|
||||
|
||||
Each table with location data has:
|
||||
- `location` (POINT SRID 4326): Spatial column for efficient geographical queries
|
||||
- `latitude` (DECIMAL(10,8)): Decimal latitude for easy access
|
||||
- `longitude` (DECIMAL(11,8)): Decimal longitude for easy access
|
||||
|
||||
## Schema Models
|
||||
|
||||
### Users
|
||||
- Clerk-based authentication (clerkId field)
|
||||
- Profile information synced from Clerk
|
||||
- App-specific data: impact score, notification preferences, areas of interest
|
||||
|
||||
### Reports
|
||||
- Litter reports with photos and GPS coordinates
|
||||
- Spatial indexing for location-based queries
|
||||
- Verification status tracking
|
||||
- Environmental concerns linked to reports
|
||||
|
||||
### Events
|
||||
- Cleanup events with location and scheduling
|
||||
- Participant registration tracking
|
||||
- Completion data (litter collected, photos)
|
||||
|
||||
### Verifications
|
||||
- Community-driven report verification
|
||||
- Tracks verify/dispute actions
|
||||
- Audit trail with timestamps
|
||||
|
||||
### Environmental Concerns
|
||||
- Additional environmental issues beyond litter
|
||||
- Linked to reports
|
||||
- Severity levels and categorization
|
||||
|
||||
### Hotspots
|
||||
- Calculated areas with high litter concentration
|
||||
- Spatial queries to identify clusters
|
||||
- Severity scoring based on report frequency
|
||||
|
||||
## Indexes
|
||||
|
||||
### Standard Indexes
|
||||
- Primary keys on all tables (UUID)
|
||||
- Foreign key indexes for relationships
|
||||
- Unique indexes on email, username, clerk_id
|
||||
- Date indexes for time-based queries
|
||||
- Enum indexes for filtering (litter_type, status, etc.)
|
||||
|
||||
### Spatial Indexes
|
||||
- `idx_location` on reports.location
|
||||
- `idx_location` on events.location
|
||||
- `idx_location` on hotspots.location
|
||||
|
||||
These spatial indexes enable efficient queries like:
|
||||
- Finding reports within a radius
|
||||
- Identifying nearby events
|
||||
- Calculating hotspot clusters
|
||||
|
||||
## Migrations
|
||||
|
||||
### Running Migrations
|
||||
|
||||
```bash
|
||||
# Apply pending migrations
|
||||
npm run prisma:migrate
|
||||
|
||||
# Deploy migrations (production)
|
||||
npx prisma migrate deploy
|
||||
|
||||
# Create a new migration
|
||||
npx prisma migrate dev --name migration_name
|
||||
```
|
||||
|
||||
### Migration History
|
||||
|
||||
1. **20241116000000_initial_setup**: Initial database schema with all tables
|
||||
2. **20241116000001_add_clerk_auth**: Added Clerk authentication support (clerk_id field)
|
||||
|
||||
## Prisma Client
|
||||
|
||||
### Generate Client
|
||||
|
||||
```bash
|
||||
npm run prisma:generate
|
||||
```
|
||||
|
||||
### Usage Example
|
||||
|
||||
```typescript
|
||||
import { PrismaClient } from '@prisma/client';
|
||||
|
||||
const prisma = new PrismaClient();
|
||||
|
||||
// Create a report with spatial data
|
||||
const report = await prisma.report.create({
|
||||
data: {
|
||||
userId: 'user-uuid',
|
||||
latitude: 53.3498,
|
||||
longitude: -6.2603,
|
||||
// Note: location POINT field is handled by database triggers or raw queries
|
||||
locationSource: 'gps',
|
||||
litterType: 'plastic',
|
||||
quantity: 'moderate',
|
||||
photoUrls: ['https://...'],
|
||||
},
|
||||
});
|
||||
|
||||
// Find reports near a location (requires raw SQL for spatial queries)
|
||||
const nearbyReports = await prisma.$queryRaw`
|
||||
SELECT id, latitude, longitude,
|
||||
ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText('POINT(${longitude} ${latitude})', 4326)
|
||||
) as distance
|
||||
FROM reports
|
||||
WHERE ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText('POINT(${longitude} ${latitude})', 4326)
|
||||
) <= ${radiusInMeters}
|
||||
ORDER BY distance
|
||||
`;
|
||||
```
|
||||
|
||||
## Spatial Query Examples
|
||||
|
||||
### Find Reports Within Radius
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
FROM reports
|
||||
WHERE ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText('POINT(-6.2603 53.3498)', 4326)
|
||||
) <= 5000; -- 5km radius
|
||||
```
|
||||
|
||||
### Calculate Hotspots
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
ST_AsText(ST_Centroid(ST_Collect(location))) as center,
|
||||
COUNT(*) as report_count,
|
||||
AVG(latitude) as avg_lat,
|
||||
AVG(longitude) as avg_lng
|
||||
FROM reports
|
||||
WHERE created_at >= DATE_SUB(NOW(), INTERVAL 30 DAY)
|
||||
GROUP BY ST_GeoHash(location, 7)
|
||||
HAVING COUNT(*) >= 5;
|
||||
```
|
||||
|
||||
### Find Nearest Event
|
||||
|
||||
```sql
|
||||
SELECT id, title, location_name,
|
||||
ST_Distance_Sphere(
|
||||
location,
|
||||
ST_GeomFromText('POINT(-6.2603 53.3498)', 4326)
|
||||
) as distance
|
||||
FROM events
|
||||
WHERE status = 'upcoming'
|
||||
ORDER BY distance
|
||||
LIMIT 1;
|
||||
```
|
||||
|
||||
## Database Maintenance
|
||||
|
||||
### Backup
|
||||
|
||||
```bash
|
||||
mysqldump -h 192.168.4.20 -u beach_litter_user -p beach_litter_mapping > backup.sql
|
||||
```
|
||||
|
||||
### Restore
|
||||
|
||||
```bash
|
||||
mysql -h 192.168.4.20 -u beach_litter_user -p beach_litter_mapping < backup.sql
|
||||
```
|
||||
|
||||
### Prisma Studio
|
||||
|
||||
View and edit data in a GUI:
|
||||
|
||||
```bash
|
||||
npm run prisma:studio
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Spatial Indexes**: All location columns have spatial indexes for fast queries
|
||||
2. **Composite Indexes**: lat/lng pairs indexed for non-spatial queries
|
||||
3. **Date Indexes**: created_at, scheduled_date indexed for time-based filtering
|
||||
4. **Foreign Key Indexes**: All foreign keys automatically indexed
|
||||
5. **Enum Indexes**: litter_type, status, verification_status indexed for filtering
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Issues
|
||||
|
||||
If you can't connect to the database:
|
||||
1. Check network connectivity to 192.168.4.20
|
||||
2. Verify MySQL is running on port 3306
|
||||
3. Confirm user credentials in .env file
|
||||
4. Check firewall rules
|
||||
|
||||
### Migration Errors
|
||||
|
||||
If migrations fail:
|
||||
1. Check database schema with `npx prisma db pull`
|
||||
2. Resolve conflicts manually if needed
|
||||
3. Mark migrations as applied: `npx prisma migrate resolve --applied migration_name`
|
||||
|
||||
### Spatial Data Issues
|
||||
|
||||
If spatial queries fail:
|
||||
1. Verify MySQL version is 8.0+
|
||||
2. Check that spatial indexes exist: `SHOW INDEX FROM reports`
|
||||
3. Ensure SRID 4326 is used consistently
|
||||
4. Use ST_GeomFromText() for creating POINT values
|
||||
|
||||
## References
|
||||
|
||||
- [Prisma Documentation](https://www.prisma.io/docs)
|
||||
- [MySQL Spatial Data Types](https://dev.mysql.com/doc/refman/8.0/en/spatial-types.html)
|
||||
- [MySQL Spatial Functions](https://dev.mysql.com/doc/refman/8.0/en/spatial-function-reference.html)
|
||||
- [Clerk Authentication](https://clerk.com/docs)
|
||||
@@ -0,0 +1,155 @@
|
||||
-- CreateTable
|
||||
CREATE TABLE `users` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`clerk_id` VARCHAR(255) NOT NULL,
|
||||
`email` VARCHAR(255) NOT NULL,
|
||||
`username` VARCHAR(100) NOT NULL,
|
||||
`created_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
`first_name` VARCHAR(100) NULL,
|
||||
`last_name` VARCHAR(100) NULL,
|
||||
`avatar_url` VARCHAR(500) NULL,
|
||||
`impact_score` INTEGER NULL DEFAULT 0,
|
||||
`notification_email` BOOLEAN NULL DEFAULT true,
|
||||
`notification_in_app` BOOLEAN NULL DEFAULT true,
|
||||
`areas_of_interest` JSON NULL,
|
||||
|
||||
UNIQUE INDEX `clerk_id`(`clerk_id`),
|
||||
UNIQUE INDEX `email`(`email`),
|
||||
UNIQUE INDEX `username`(`username`),
|
||||
INDEX `idx_clerk_id`(`clerk_id`),
|
||||
INDEX `idx_email`(`email`),
|
||||
INDEX `idx_username`(`username`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `reports` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`user_id` VARCHAR(36) NOT NULL,
|
||||
`location` POINT NOT NULL SRID 4326,
|
||||
`latitude` DECIMAL(10, 8) NOT NULL,
|
||||
`longitude` DECIMAL(11, 8) NOT NULL,
|
||||
`location_source` ENUM('exif', 'gps', 'manual') NOT NULL,
|
||||
`photo_urls` JSON NOT NULL,
|
||||
`photo_timestamp` TIMESTAMP(0) NULL,
|
||||
`litter_type` ENUM('plastic', 'metal', 'glass', 'organic', 'hazardous', 'other') NOT NULL,
|
||||
`quantity` ENUM('minimal', 'moderate', 'significant', 'severe') NOT NULL,
|
||||
`description` TEXT NULL,
|
||||
`created_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
`submitted_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`verification_status` ENUM('pending', 'verified', 'disputed') NULL DEFAULT 'pending',
|
||||
|
||||
INDEX `idx_created_at`(`created_at`),
|
||||
INDEX `idx_lat_lng`(`latitude`, `longitude`),
|
||||
INDEX `idx_litter_type`(`litter_type`),
|
||||
SPATIAL INDEX `idx_location`(`location`),
|
||||
INDEX `user_id`(`user_id`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `events` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`organizer_id` VARCHAR(36) NOT NULL,
|
||||
`title` VARCHAR(255) NOT NULL,
|
||||
`description` TEXT NOT NULL,
|
||||
`location` POINT NOT NULL SRID 4326,
|
||||
`latitude` DECIMAL(10, 8) NOT NULL,
|
||||
`longitude` DECIMAL(11, 8) NOT NULL,
|
||||
`location_name` VARCHAR(255) NOT NULL,
|
||||
`scheduled_date` TIMESTAMP(0) NOT NULL,
|
||||
`duration` INTEGER NOT NULL COMMENT 'Duration in minutes',
|
||||
`status` ENUM('upcoming', 'completed', 'cancelled') NULL DEFAULT 'upcoming',
|
||||
`participant_count` INTEGER NULL DEFAULT 0,
|
||||
`litter_collected` DECIMAL(10, 2) NULL COMMENT 'Kilograms collected',
|
||||
`photos` JSON NULL,
|
||||
`created_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
|
||||
SPATIAL INDEX `idx_location`(`location`),
|
||||
INDEX `idx_scheduled_date`(`scheduled_date`),
|
||||
INDEX `idx_status`(`status`),
|
||||
INDEX `organizer_id`(`organizer_id`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `event_registrations` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`user_id` VARCHAR(36) NOT NULL,
|
||||
`event_id` VARCHAR(36) NOT NULL,
|
||||
`registered_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`attended` BOOLEAN NULL DEFAULT false,
|
||||
|
||||
INDEX `event_id`(`event_id`),
|
||||
UNIQUE INDEX `unique_user_event`(`user_id`, `event_id`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `verifications` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`report_id` VARCHAR(36) NOT NULL,
|
||||
`user_id` VARCHAR(36) NOT NULL,
|
||||
`verification_type` ENUM('verify', 'dispute') NOT NULL,
|
||||
`comment` TEXT NULL,
|
||||
`created_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
INDEX `user_id`(`user_id`),
|
||||
UNIQUE INDEX `unique_report_user`(`report_id`, `user_id`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `environmental_concerns` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`report_id` VARCHAR(36) NOT NULL,
|
||||
`concern_type` VARCHAR(100) NOT NULL COMMENT 'water_quality, erosion, wildlife_impact, algae_bloom, etc.',
|
||||
`severity` ENUM('low', 'medium', 'high', 'critical') NOT NULL,
|
||||
`description` TEXT NOT NULL,
|
||||
`created_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
INDEX `idx_concern_type`(`concern_type`),
|
||||
INDEX `report_id`(`report_id`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE `hotspots` (
|
||||
`id` VARCHAR(36) NOT NULL DEFAULT (UUID()),
|
||||
`location` POINT NOT NULL SRID 4326,
|
||||
`latitude` DECIMAL(10, 8) NOT NULL,
|
||||
`longitude` DECIMAL(11, 8) NOT NULL,
|
||||
`radius` DECIMAL(10, 2) NOT NULL COMMENT 'Radius in meters',
|
||||
`report_count` INTEGER NOT NULL,
|
||||
`severity_score` DECIMAL(5, 2) NOT NULL,
|
||||
`last_report_date` TIMESTAMP(0) NOT NULL,
|
||||
`calculated_at` TIMESTAMP(0) NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
SPATIAL INDEX `idx_location`(`location`),
|
||||
INDEX `idx_severity_score`(`severity_score`),
|
||||
PRIMARY KEY (`id`)
|
||||
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE=InnoDB;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `reports` ADD CONSTRAINT `reports_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `events` ADD CONSTRAINT `events_ibfk_1` FOREIGN KEY (`organizer_id`) REFERENCES `users`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `event_registrations` ADD CONSTRAINT `event_registrations_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `event_registrations` ADD CONSTRAINT `event_registrations_ibfk_2` FOREIGN KEY (`event_id`) REFERENCES `events`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `verifications` ADD CONSTRAINT `verifications_ibfk_1` FOREIGN KEY (`report_id`) REFERENCES `reports`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `verifications` ADD CONSTRAINT `verifications_ibfk_2` FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE `environmental_concerns` ADD CONSTRAINT `environmental_concerns_ibfk_1` FOREIGN KEY (`report_id`) REFERENCES `reports`(`id`) ON DELETE CASCADE ON UPDATE NO ACTION;
|
||||
@@ -0,0 +1,13 @@
|
||||
-- AlterTable: Add clerk_id column and remove password_hash
|
||||
ALTER TABLE `users` ADD COLUMN `clerk_id` VARCHAR(255) NULL;
|
||||
|
||||
-- Create unique index on clerk_id
|
||||
CREATE UNIQUE INDEX `clerk_id` ON `users`(`clerk_id`);
|
||||
|
||||
-- Create index for clerk_id lookups
|
||||
CREATE INDEX `idx_clerk_id` ON `users`(`clerk_id`);
|
||||
|
||||
-- Drop password_hash column (after data migration if needed)
|
||||
-- Note: In production, you would first migrate existing users to Clerk before dropping this column
|
||||
-- For now, we'll keep it to avoid data loss, but it should be removed after Clerk migration is complete
|
||||
-- ALTER TABLE `users` DROP COLUMN `password_hash`;
|
||||
3
server/prisma/migrations/migration_lock.toml
Normal file
3
server/prisma/migrations/migration_lock.toml
Normal file
@@ -0,0 +1,3 @@
|
||||
# Please do not edit this file manually
|
||||
# It should be added in your version-control system (i.e. Git)
|
||||
provider = "mysql"
|
||||
@@ -21,11 +21,13 @@ model User {
|
||||
notificationEmail Boolean? @default(true) @map("notification_email")
|
||||
notificationInApp Boolean? @default(true) @map("notification_in_app")
|
||||
areasOfInterest Json? @map("areas_of_interest")
|
||||
clerkId String? @unique(map: "clerk_id") @map("clerk_id") @db.VarChar(255)
|
||||
eventRegistrations EventRegistration[]
|
||||
organizedEvents Event[] @relation("EventOrganizer")
|
||||
reports Report[]
|
||||
verifications Verification[]
|
||||
|
||||
@@index([clerkId], map: "idx_clerk_id")
|
||||
@@index([email], map: "idx_email")
|
||||
@@index([username], map: "idx_username")
|
||||
@@map("users")
|
||||
|
||||
Reference in New Issue
Block a user