ocean-cleanup/server/EXTERNAL_API_SETUP.md

# External API Setup Guide

This guide explains how to obtain API keys for the external data integrations and configure them in the Beach Litter Mapping Platform.

## Overview

The platform integrates with several external APIs to provide environmental context data:

- **OpenWeatherMap** - Current weather conditions
- **WorldTides** - Tide predictions and times
- **EPA Ireland** - Water quality data (placeholder)
- **Blue Flag/Beach Quality** - Beach ratings (placeholder)

## Required API Keys

### 1. OpenWeatherMap API (Weather Data)

**Purpose:** Provides current weather conditions including temperature, humidity, wind speed, and weather descriptions.

**How to Get:**

1. Go to [OpenWeatherMap](https://openweathermap.org/)
2. Click "Sign Up" in the top right corner
3. Create a free account
4. After signing in, go to your [API Keys page](https://home.openweathermap.org/api_keys)
5. Copy your default API key (or generate a new one)

**Free Tier Limits:**
- 1,000 API calls per day
- 60 calls per minute
- Current weather data included

**Cost:** Free tier available, paid plans start at $40/month for higher limits

**Add to `.env`:**
```env
OPENWEATHER_API_KEY="your_api_key_here"
```

**Documentation:** https://openweathermap.org/api

---

### 2. Tide Data (Multiple Free Options)

**Purpose:** Provides tide predictions, high/low tide times, and tide heights for coastal locations.

**Current Implementation:** Mock data for demonstration purposes

**Free Alternatives:**

#### Option A: Stormglass.io (Recommended for Ireland)
- **Free Tier:** 10 requests per day
- **Coverage:** Global including Ireland
- **How to Get:**
  1. Go to [Stormglass.io](https://stormglass.io/)
  2. Sign up for free account
  3. Get API key from dashboard
- **Documentation:** https://docs.stormglass.io/
- **Cost:** Free tier: 10 req/day, Paid: $9/month for 10,000 requests

#### Option B: NOAA Tides & Currents (US Only)
- **Free:** Unlimited requests
- **Coverage:** US coastal locations only
- **No API key required**
- **URL:** https://api.tidesandcurrents.noaa.gov/api/prod/
- **Good for:** Testing if you're in the US

#### Option C: UK Hydrographic Office (UK Only)
- **Free:** For UK locations
- **Coverage:** UK coastal areas
- **URL:** https://admiraltyapi.azure-api.net/
- **Good for:** If expanding to UK beaches

#### Option D: Static Tide Tables
- **Free:** No API needed
- **Method:** Create a database of Irish tide stations with pre-calculated tables
- **Source:** Irish Marine Institute or manual entry
- **Pros:** No API limits, always available
- **Cons:** Requires manual updates, less accurate

**Current Setup:**
The system currently returns mock tide data for demonstration. To enable real data:

1. Sign up for Stormglass.io free tier
2. Add API key to `.env`:
```env
STORMGLASS_API_KEY="your_api_key_here"
```
3. Update `externalDataService.ts` to use Stormglass API

**Note:** With only 10 free requests per day, consider:
- Caching aggressively (tide data changes slowly)
- Using for specific locations only
- Implementing static tide tables for common beaches

---

### 3. Caching

**Current Implementation:** In-memory caching (no external service required)

**How it works:**
- API responses cached for 6 hours
- Automatic cache expiration
- No setup required

**Optional: Redis for Production**

If you want persistent caching across server restarts:

**Docker:**
```bash
docker run -d -p 6379:6379 --name redis redis:alpine
```

**Add to `.env`:**
```env
REDIS_URL="redis://localhost:6379"
```

Then update `src/config/redis.ts` to use Redis client instead of in-memory cache.

**Note:** For most use cases, in-memory caching is sufficient.

---

## Optional APIs (Placeholder Implementations)

These APIs are not fully implemented yet but have placeholder endpoints:

### 4. EPA Ireland Water Quality Data

**Purpose:** Water quality monitoring data for Irish beaches and coastal areas.

**Source:** [EPA Ireland Open Data Portal](https://gis.epa.ie/EPAMaps/)

**Status:** Placeholder implementation - returns mock data
**Cost:** Free (public data)
**API Key:** Not required

**To Implement:**
- Research EPA's data access methods (may require web scraping or specific API)
- Contact EPA for API access if available
- Alternative: Use EU Bathing Water Quality data

---

### 5. Beach Quality Ratings (Blue Flag, Green Coast)

**Purpose:** Beach quality certifications and awards.

**Sources:**
- [Blue Flag Ireland](https://www.blueflagireland.org/)
- [Green Coast Awards](https://www.greencoastawards.ie/)
- [An Taisce](https://www.antaisce.org/)

**Status:** Placeholder implementation - returns mock data
**Cost:** Free (public data)
**API Key:** Not required

**To Implement:**
- These organizations may not have public APIs
- May require web scraping or manual data entry
- Could create a static database of certified beaches

---

## Configuration

### Complete `.env` File Example

```env
# Database
DATABASE_URL="mysql://beach_litter_user:BeachClean2024!@192.168.4.20:3306/beach_litter_mapping"

# Clerk Authentication
CLERK_SECRET_KEY="sk_test_your_secret_key_here"
CLERK_WEBHOOK_SECRET="whsec_your_webhook_secret_here"

# Server
PORT=3000
NODE_ENV=development

# File Upload
UPLOAD_DIR=./uploads
MAX_FILE_SIZE=5242880

# Redis Cache
REDIS_URL="redis://localhost:6379"

# External APIs
OPENWEATHER_API_KEY="your_openweather_api_key_here"
WORLDTIDES_API_KEY="your_worldtides_api_key_here"
```

---

## Testing the Integration

After configuring your API keys, test the integration:

```bash
# Start Redis (if not already running)
redis-cli ping

# Run the test script
npm run dev
# In another terminal:
cd server
npx tsx src/scripts/testExternalAPIs.ts
```

Or test via HTTP:

```bash
# Test weather endpoint
curl http://localhost:3005/api/external/weather/53.3498/-6.2603

# Test tide endpoint
curl http://localhost:3005/api/external/tides/53.3498/-6.2603

# Test all data
curl http://localhost:3005/api/external/all/53.3498/-6.2603
```

---

## Cache Behavior

- **Cache Duration:** 6 hours (21,600 seconds)
- **Cache Keys:** Based on coordinates rounded to 4 decimal places
- **Fallback:** If Redis is unavailable, API calls work without caching (degraded mode)

**Cache Key Format:**
```
weather:53.3498:-6.2603
tide:53.3498:-6.2603
water-quality:53.3498:-6.2603
beach-quality:53.3498:-6.2603
```

---

## Cost Optimization Tips

1. **Use Redis Caching:** Reduces API calls by 6 hours per location
2. **Round Coordinates:** Cache keys use 4 decimal places (~11m precision)
3. **Monitor Usage:** Check API dashboards regularly
4. **Rate Limiting:** Implement rate limiting on frontend to prevent abuse
5. **Lazy Loading:** Only fetch data when user views environmental panel

---

## Troubleshooting

### Weather/Tide Data Returns 503

**Cause:** API key not configured or invalid

**Solution:**
1. Check `.env` file has correct API keys
2. Verify API keys are active in provider dashboards
3. Check API key hasn't exceeded rate limits
4. Restart server after adding keys

### Redis Connection Errors

**Cause:** Redis not running or wrong connection URL

**Solution:**
1. Verify Redis is running: `redis-cli ping`
2. Check `REDIS_URL` in `.env`
3. For Docker: ensure container is running
4. Check firewall isn't blocking port 6379

### Data Not Updating

**Cause:** Cached data still valid

**Solution:**
1. Wait for cache to expire (6 hours)
2. Clear cache manually: `redis-cli FLUSHALL`
3. Use different coordinates to bypass cache

---

## Production Deployment

### Environment Variables

Set these in your production environment:

```env
NODE_ENV=production
REDIS_URL="redis://your-production-redis-url"
OPENWEATHER_API_KEY="your_production_key"
WORLDTIDES_API_KEY="your_production_key"
```

### Recommended Services

**Redis Hosting:**
- **Upstash** (Recommended for serverless): https://upstash.com/
- **Redis Cloud**: https://redis.com/try-free/
- **Railway**: https://railway.app/

**API Key Management:**
- Store keys in environment variables (never commit to git)
- Use secrets management (AWS Secrets Manager, etc.)
- Rotate keys periodically
- Monitor usage and set up alerts

---

## Future Enhancements

1. **EPA Integration:** Implement real EPA water quality API
2. **Beach Database:** Create static database of Irish beaches with ratings
3. **Marine Biodiversity:** Integrate with GBIF or similar databases
4. **Forecast Data:** Add weather forecasts (3-7 days)
5. **Historical Data:** Store and display historical trends
6. **Alert System:** Notify users of poor water quality or weather warnings

---

## Support

- **OpenWeatherMap Support:** https://openweathermap.org/faq
- **WorldTides Support:** support@worldtides.info
- **Redis Documentation:** https://redis.io/docs/