Admin/ocean-cleanup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
ocean-cleanup/server/EXTERNAL_API_SETUP.md
Paul Bailey e0889766cc docs: add water quality integration guide - no API key required 
- Document EU Bathing Water Quality API (free, no key needed)
- Add EPA Ireland and Marine Institute alternatives
- Provide implementation examples
- Clarify that water quality data is completely free and public
2025-11-16 23:30:54 +00:00

10 KiB
Raw Permalink Blame History

External API Setup Guide

This guide explains how to obtain API keys for the external data integrations and configure them in the Beach Litter Mapping Platform.

Overview

The platform integrates with several external APIs to provide environmental context data:

  • OpenWeatherMap - Current weather conditions
  • WorldTides - Tide predictions and times
  • EPA Ireland - Water quality data (placeholder)
  • Blue Flag/Beach Quality - Beach ratings (placeholder)

Required API Keys

1. OpenWeatherMap API (Weather Data)

Purpose: Provides current weather conditions including temperature, humidity, wind speed, and weather descriptions.

How to Get:

  1. Go to OpenWeatherMap
  2. Click "Sign Up" in the top right corner
  3. Create a free account
  4. After signing in, go to your API Keys page
  5. Copy your default API key (or generate a new one)

Free Tier Limits:

  • 1,000 API calls per day
  • 60 calls per minute
  • Current weather data included

Cost: Free tier available, paid plans start at $40/month for higher limits

Add to .env:

OPENWEATHER_API_KEY="your_api_key_here"

Documentation: https://openweathermap.org/api

2. Tide Data (Multiple Free Options)

Purpose: Provides tide predictions, high/low tide times, and tide heights for coastal locations.

Current Implementation: Mock data for demonstration purposes

Free Alternatives:

Option A: Stormglass.io (Recommended for Ireland)

  • Free Tier: 10 requests per day
  • Coverage: Global including Ireland
  • How to Get:
    1. Go to Stormglass.io
    2. Sign up for free account
    3. Get API key from dashboard
  • Documentation: https://docs.stormglass.io/
  • Cost: Free tier: 10 req/day, Paid: $9/month for 10,000 requests

Option B: NOAA Tides & Currents (US Only)

Option C: UK Hydrographic Office (UK Only)

Option D: Static Tide Tables

  • Free: No API needed
  • Method: Create a database of Irish tide stations with pre-calculated tables
  • Source: Irish Marine Institute or manual entry
  • Pros: No API limits, always available
  • Cons: Requires manual updates, less accurate

Current Setup: The system currently returns mock tide data for demonstration. To enable real data:

  1. Sign up for Stormglass.io free tier
  2. Add API key to .env:
STORMGLASS_API_KEY="your_api_key_here"
  1. Update externalDataService.ts to use Stormglass API

Note: With only 10 free requests per day, consider:

  • Caching aggressively (tide data changes slowly)
  • Using for specific locations only
  • Implementing static tide tables for common beaches

3. Caching

Current Implementation: In-memory caching (no external service required)

How it works:

  • API responses cached for 6 hours
  • Automatic cache expiration
  • No setup required

Optional: Redis for Production

If you want persistent caching across server restarts:

Docker:

docker run -d -p 6379:6379 --name redis redis:alpine

Add to .env:

REDIS_URL="redis://localhost:6379"

Then update src/config/redis.ts to use Redis client instead of in-memory cache.

Note: For most use cases, in-memory caching is sufficient.

Optional APIs (Placeholder Implementations)

These APIs are not fully implemented yet but have placeholder endpoints:

4. EPA Ireland Water Quality Data

Purpose: Water quality monitoring data for Irish beaches and coastal areas.

Available Data Sources:

Option A: EU Bathing Water Quality (Recommended)

How to Access:

  1. Use EEA's REST API endpoint
  2. Filter by country code "IE" for Ireland
  3. Parse XML/JSON response
  4. No authentication required

Example Endpoint:

https://www.eea.europa.eu/api/EMIS/get_bathing_water_data?country=IE

Option B: EPA Ireland GeoPortal

  • Source: EPA Ireland
  • Portal: https://gis.epa.ie/EPAMaps/
  • Cost: Free, no API key required
  • Method: Web scraping or manual data download
  • Data: Real-time and historical water quality data

Access Methods:

  1. Manual Download: Download datasets from EPA GeoPortal
  2. Web Scraping: Parse EPA maps (requires permission)
  3. Contact EPA: Request API access at info@epa.ie

Option C: Marine Institute Ireland

  • Source: Marine Institute
  • Portal: https://data.marine.ie/
  • Cost: Free, no API key required
  • Data: Marine water quality, oceanographic data
  • Format: Various formats including API access

How to Access:

  1. Browse datasets at https://data.marine.ie/
  2. Use ERDDAP server for programmatic access
  3. No API key required for public data

Option D: Static Database (Simplest)

  • Method: Create a database of Irish beaches with latest quality ratings
  • Source: Manual entry from EPA/EU reports
  • Update: Quarterly or annually
  • Pros: Fast, reliable, no external dependencies
  • Cons: Requires manual updates

Recommended Approach: Use EU Bathing Water Quality API (Option A) as it's:

  • Free and public
  • No API key required
  • Covers all Irish beaches
  • Regularly updated
  • Easy to integrate

Status: Placeholder implementation - needs integration with one of the above sources API Key: Not required for any option

5. Beach Quality Ratings (Blue Flag, Green Coast)

Purpose: Beach quality certifications and awards.

Sources:

Status: Placeholder implementation - returns mock data Cost: Free (public data) API Key: Not required

To Implement:

  • These organizations may not have public APIs
  • May require web scraping or manual data entry
  • Could create a static database of certified beaches

Configuration

Complete .env File Example

# Database
DATABASE_URL="mysql://beach_litter_user:BeachClean2024!@192.168.4.20:3306/beach_litter_mapping"

# Clerk Authentication
CLERK_SECRET_KEY="sk_test_your_secret_key_here"
CLERK_WEBHOOK_SECRET="whsec_your_webhook_secret_here"

# Server
PORT=3000
NODE_ENV=development

# File Upload
UPLOAD_DIR=./uploads
MAX_FILE_SIZE=5242880

# Redis Cache
REDIS_URL="redis://localhost:6379"

# External APIs
OPENWEATHER_API_KEY="your_openweather_api_key_here"
WORLDTIDES_API_KEY="your_worldtides_api_key_here"

Testing the Integration

After configuring your API keys, test the integration:

# Start Redis (if not already running)
redis-cli ping

# Run the test script
npm run dev
# In another terminal:
cd server
npx tsx src/scripts/testExternalAPIs.ts

Or test via HTTP:

# Test weather endpoint
curl http://localhost:3005/api/external/weather/53.3498/-6.2603

# Test tide endpoint
curl http://localhost:3005/api/external/tides/53.3498/-6.2603

# Test all data
curl http://localhost:3005/api/external/all/53.3498/-6.2603

Cache Behavior

  • Cache Duration: 6 hours (21,600 seconds)
  • Cache Keys: Based on coordinates rounded to 4 decimal places
  • Fallback: If Redis is unavailable, API calls work without caching (degraded mode)

Cache Key Format:

weather:53.3498:-6.2603
tide:53.3498:-6.2603
water-quality:53.3498:-6.2603
beach-quality:53.3498:-6.2603

Cost Optimization Tips

  1. Use Redis Caching: Reduces API calls by 6 hours per location
  2. Round Coordinates: Cache keys use 4 decimal places (~11m precision)
  3. Monitor Usage: Check API dashboards regularly
  4. Rate Limiting: Implement rate limiting on frontend to prevent abuse
  5. Lazy Loading: Only fetch data when user views environmental panel

Troubleshooting

Weather/Tide Data Returns 503

Cause: API key not configured or invalid

Solution:

  1. Check .env file has correct API keys
  2. Verify API keys are active in provider dashboards
  3. Check API key hasn't exceeded rate limits
  4. Restart server after adding keys

Redis Connection Errors

Cause: Redis not running or wrong connection URL

Solution:

  1. Verify Redis is running: redis-cli ping
  2. Check REDIS_URL in .env
  3. For Docker: ensure container is running
  4. Check firewall isn't blocking port 6379

Data Not Updating

Cause: Cached data still valid

Solution:

  1. Wait for cache to expire (6 hours)
  2. Clear cache manually: redis-cli FLUSHALL
  3. Use different coordinates to bypass cache

Production Deployment

Environment Variables

Set these in your production environment:

NODE_ENV=production
REDIS_URL="redis://your-production-redis-url"
OPENWEATHER_API_KEY="your_production_key"
WORLDTIDES_API_KEY="your_production_key"

Recommended Services

Redis Hosting:

API Key Management:

  • Store keys in environment variables (never commit to git)
  • Use secrets management (AWS Secrets Manager, etc.)
  • Rotate keys periodically
  • Monitor usage and set up alerts

Future Enhancements

  1. EPA Integration: Implement real EPA water quality API
  2. Beach Database: Create static database of Irish beaches with ratings
  3. Marine Biodiversity: Integrate with GBIF or similar databases
  4. Forecast Data: Add weather forecasts (3-7 days)
  5. Historical Data: Store and display historical trends
  6. Alert System: Notify users of poor water quality or weather warnings

Support

Reference in New Issue View Git Blame Copy Permalink