Paul Bailey
e0889766cc
- Document EU Bathing Water Quality API (free, no key needed) - Add EPA Ireland and Marine Institute alternatives - Provide implementation examples - Clarify that water quality data is completely free and public
10 KiB
External API Setup Guide
This guide explains how to obtain API keys for the external data integrations and configure them in the Beach Litter Mapping Platform.
Overview
The platform integrates with several external APIs to provide environmental context data:
- OpenWeatherMap - Current weather conditions
- WorldTides - Tide predictions and times
- EPA Ireland - Water quality data (placeholder)
- Blue Flag/Beach Quality - Beach ratings (placeholder)
Required API Keys
1. OpenWeatherMap API (Weather Data)
Purpose: Provides current weather conditions including temperature, humidity, wind speed, and weather descriptions.
How to Get:
- Go to OpenWeatherMap
- Click "Sign Up" in the top right corner
- Create a free account
- After signing in, go to your API Keys page
- Copy your default API key (or generate a new one)
Free Tier Limits:
- 1,000 API calls per day
- 60 calls per minute
- Current weather data included
Cost: Free tier available, paid plans start at $40/month for higher limits
Add to .env:
OPENWEATHER_API_KEY="your_api_key_here"
Documentation: https://openweathermap.org/api
2. Tide Data (Multiple Free Options)
Purpose: Provides tide predictions, high/low tide times, and tide heights for coastal locations.
Current Implementation: Mock data for demonstration purposes
Free Alternatives:
Option A: Stormglass.io (Recommended for Ireland)
- Free Tier: 10 requests per day
- Coverage: Global including Ireland
- How to Get:
- Go to Stormglass.io
- Sign up for free account
- Get API key from dashboard
- Documentation: https://docs.stormglass.io/
- Cost: Free tier: 10 req/day, Paid: $9/month for 10,000 requests
Option B: NOAA Tides & Currents (US Only)
- Free: Unlimited requests
- Coverage: US coastal locations only
- No API key required
- URL: https://api.tidesandcurrents.noaa.gov/api/prod/
- Good for: Testing if you're in the US
Option C: UK Hydrographic Office (UK Only)
- Free: For UK locations
- Coverage: UK coastal areas
- URL: https://admiraltyapi.azure-api.net/
- Good for: If expanding to UK beaches
Option D: Static Tide Tables
- Free: No API needed
- Method: Create a database of Irish tide stations with pre-calculated tables
- Source: Irish Marine Institute or manual entry
- Pros: No API limits, always available
- Cons: Requires manual updates, less accurate
Current Setup: The system currently returns mock tide data for demonstration. To enable real data:
- Sign up for Stormglass.io free tier
- Add API key to
.env:
STORMGLASS_API_KEY="your_api_key_here"
- Update
externalDataService.tsto use Stormglass API
Note: With only 10 free requests per day, consider:
- Caching aggressively (tide data changes slowly)
- Using for specific locations only
- Implementing static tide tables for common beaches
3. Caching
Current Implementation: In-memory caching (no external service required)
How it works:
- API responses cached for 6 hours
- Automatic cache expiration
- No setup required
Optional: Redis for Production
If you want persistent caching across server restarts:
Docker:
docker run -d -p 6379:6379 --name redis redis:alpine
Add to .env:
REDIS_URL="redis://localhost:6379"
Then update src/config/redis.ts to use Redis client instead of in-memory cache.
Note: For most use cases, in-memory caching is sufficient.
Optional APIs (Placeholder Implementations)
These APIs are not fully implemented yet but have placeholder endpoints:
4. EPA Ireland Water Quality Data
Purpose: Water quality monitoring data for Irish beaches and coastal areas.
Available Data Sources:
Option A: EU Bathing Water Quality (Recommended)
- Source: European Environment Agency
- API: https://www.eea.europa.eu/data-and-maps/data/bathing-water-directive-status-of-bathing-water
- Cost: Free, no API key required
- Coverage: All EU designated bathing waters including Ireland
- Data Format: CSV/XML downloads or REST API
- Update Frequency: Annual with in-season updates
- Documentation: https://www.eea.europa.eu/help/glossary/eea-glossary/bathing-water
How to Access:
- Use EEA's REST API endpoint
- Filter by country code "IE" for Ireland
- Parse XML/JSON response
- No authentication required
Example Endpoint:
https://www.eea.europa.eu/api/EMIS/get_bathing_water_data?country=IE
Option B: EPA Ireland GeoPortal
- Source: EPA Ireland
- Portal: https://gis.epa.ie/EPAMaps/
- Cost: Free, no API key required
- Method: Web scraping or manual data download
- Data: Real-time and historical water quality data
Access Methods:
- Manual Download: Download datasets from EPA GeoPortal
- Web Scraping: Parse EPA maps (requires permission)
- Contact EPA: Request API access at info@epa.ie
Option C: Marine Institute Ireland
- Source: Marine Institute
- Portal: https://data.marine.ie/
- Cost: Free, no API key required
- Data: Marine water quality, oceanographic data
- Format: Various formats including API access
How to Access:
- Browse datasets at https://data.marine.ie/
- Use ERDDAP server for programmatic access
- No API key required for public data
Option D: Static Database (Simplest)
- Method: Create a database of Irish beaches with latest quality ratings
- Source: Manual entry from EPA/EU reports
- Update: Quarterly or annually
- Pros: Fast, reliable, no external dependencies
- Cons: Requires manual updates
Recommended Approach: Use EU Bathing Water Quality API (Option A) as it's:
- Free and public
- No API key required
- Covers all Irish beaches
- Regularly updated
- Easy to integrate
Status: Placeholder implementation - needs integration with one of the above sources API Key: Not required for any option
5. Beach Quality Ratings (Blue Flag, Green Coast)
Purpose: Beach quality certifications and awards.
Sources:
Status: Placeholder implementation - returns mock data Cost: Free (public data) API Key: Not required
To Implement:
- These organizations may not have public APIs
- May require web scraping or manual data entry
- Could create a static database of certified beaches
Configuration
Complete .env File Example
# Database
DATABASE_URL="mysql://beach_litter_user:BeachClean2024!@192.168.4.20:3306/beach_litter_mapping"
# Clerk Authentication
CLERK_SECRET_KEY="sk_test_your_secret_key_here"
CLERK_WEBHOOK_SECRET="whsec_your_webhook_secret_here"
# Server
PORT=3000
NODE_ENV=development
# File Upload
UPLOAD_DIR=./uploads
MAX_FILE_SIZE=5242880
# Redis Cache
REDIS_URL="redis://localhost:6379"
# External APIs
OPENWEATHER_API_KEY="your_openweather_api_key_here"
WORLDTIDES_API_KEY="your_worldtides_api_key_here"
Testing the Integration
After configuring your API keys, test the integration:
# Start Redis (if not already running)
redis-cli ping
# Run the test script
npm run dev
# In another terminal:
cd server
npx tsx src/scripts/testExternalAPIs.ts
Or test via HTTP:
# Test weather endpoint
curl http://localhost:3005/api/external/weather/53.3498/-6.2603
# Test tide endpoint
curl http://localhost:3005/api/external/tides/53.3498/-6.2603
# Test all data
curl http://localhost:3005/api/external/all/53.3498/-6.2603
Cache Behavior
- Cache Duration: 6 hours (21,600 seconds)
- Cache Keys: Based on coordinates rounded to 4 decimal places
- Fallback: If Redis is unavailable, API calls work without caching (degraded mode)
Cache Key Format:
weather:53.3498:-6.2603
tide:53.3498:-6.2603
water-quality:53.3498:-6.2603
beach-quality:53.3498:-6.2603
Cost Optimization Tips
- Use Redis Caching: Reduces API calls by 6 hours per location
- Round Coordinates: Cache keys use 4 decimal places (~11m precision)
- Monitor Usage: Check API dashboards regularly
- Rate Limiting: Implement rate limiting on frontend to prevent abuse
- Lazy Loading: Only fetch data when user views environmental panel
Troubleshooting
Weather/Tide Data Returns 503
Cause: API key not configured or invalid
Solution:
- Check
.envfile has correct API keys - Verify API keys are active in provider dashboards
- Check API key hasn't exceeded rate limits
- Restart server after adding keys
Redis Connection Errors
Cause: Redis not running or wrong connection URL
Solution:
- Verify Redis is running:
redis-cli ping - Check
REDIS_URLin.env - For Docker: ensure container is running
- Check firewall isn't blocking port 6379
Data Not Updating
Cause: Cached data still valid
Solution:
- Wait for cache to expire (6 hours)
- Clear cache manually:
redis-cli FLUSHALL - Use different coordinates to bypass cache
Production Deployment
Environment Variables
Set these in your production environment:
NODE_ENV=production
REDIS_URL="redis://your-production-redis-url"
OPENWEATHER_API_KEY="your_production_key"
WORLDTIDES_API_KEY="your_production_key"
Recommended Services
Redis Hosting:
- Upstash (Recommended for serverless): https://upstash.com/
- Redis Cloud: https://redis.com/try-free/
- Railway: https://railway.app/
API Key Management:
- Store keys in environment variables (never commit to git)
- Use secrets management (AWS Secrets Manager, etc.)
- Rotate keys periodically
- Monitor usage and set up alerts
Future Enhancements
- EPA Integration: Implement real EPA water quality API
- Beach Database: Create static database of Irish beaches with ratings
- Marine Biodiversity: Integrate with GBIF or similar databases
- Forecast Data: Add weather forecasts (3-7 days)
- Historical Data: Store and display historical trends
- Alert System: Notify users of poor water quality or weather warnings
Support
- OpenWeatherMap Support: https://openweathermap.org/faq
- WorldTides Support: support@worldtides.info
- Redis Documentation: https://redis.io/docs/