Paul Bailey
e0889766cc
- Document EU Bathing Water Quality API (free, no key needed) - Add EPA Ireland and Marine Institute alternatives - Provide implementation examples - Clarify that water quality data is completely free and public
386 lines
10 KiB
Markdown
386 lines
10 KiB
Markdown
# External API Setup Guide
|
|
|
|
This guide explains how to obtain API keys for the external data integrations and configure them in the Beach Litter Mapping Platform.
|
|
|
|
## Overview
|
|
|
|
The platform integrates with several external APIs to provide environmental context data:
|
|
|
|
- **OpenWeatherMap** - Current weather conditions
|
|
- **WorldTides** - Tide predictions and times
|
|
- **EPA Ireland** - Water quality data (placeholder)
|
|
- **Blue Flag/Beach Quality** - Beach ratings (placeholder)
|
|
|
|
## Required API Keys
|
|
|
|
### 1. OpenWeatherMap API (Weather Data)
|
|
|
|
**Purpose:** Provides current weather conditions including temperature, humidity, wind speed, and weather descriptions.
|
|
|
|
**How to Get:**
|
|
|
|
1. Go to [OpenWeatherMap](https://openweathermap.org/)
|
|
2. Click "Sign Up" in the top right corner
|
|
3. Create a free account
|
|
4. After signing in, go to your [API Keys page](https://home.openweathermap.org/api_keys)
|
|
5. Copy your default API key (or generate a new one)
|
|
|
|
**Free Tier Limits:**
|
|
- 1,000 API calls per day
|
|
- 60 calls per minute
|
|
- Current weather data included
|
|
|
|
**Cost:** Free tier available, paid plans start at $40/month for higher limits
|
|
|
|
**Add to `.env`:**
|
|
```env
|
|
OPENWEATHER_API_KEY="your_api_key_here"
|
|
```
|
|
|
|
**Documentation:** https://openweathermap.org/api
|
|
|
|
---
|
|
|
|
### 2. Tide Data (Multiple Free Options)
|
|
|
|
**Purpose:** Provides tide predictions, high/low tide times, and tide heights for coastal locations.
|
|
|
|
**Current Implementation:** Mock data for demonstration purposes
|
|
|
|
**Free Alternatives:**
|
|
|
|
#### Option A: Stormglass.io (Recommended for Ireland)
|
|
- **Free Tier:** 10 requests per day
|
|
- **Coverage:** Global including Ireland
|
|
- **How to Get:**
|
|
1. Go to [Stormglass.io](https://stormglass.io/)
|
|
2. Sign up for free account
|
|
3. Get API key from dashboard
|
|
- **Documentation:** https://docs.stormglass.io/
|
|
- **Cost:** Free tier: 10 req/day, Paid: $9/month for 10,000 requests
|
|
|
|
#### Option B: NOAA Tides & Currents (US Only)
|
|
- **Free:** Unlimited requests
|
|
- **Coverage:** US coastal locations only
|
|
- **No API key required**
|
|
- **URL:** https://api.tidesandcurrents.noaa.gov/api/prod/
|
|
- **Good for:** Testing if you're in the US
|
|
|
|
#### Option C: UK Hydrographic Office (UK Only)
|
|
- **Free:** For UK locations
|
|
- **Coverage:** UK coastal areas
|
|
- **URL:** https://admiraltyapi.azure-api.net/
|
|
- **Good for:** If expanding to UK beaches
|
|
|
|
#### Option D: Static Tide Tables
|
|
- **Free:** No API needed
|
|
- **Method:** Create a database of Irish tide stations with pre-calculated tables
|
|
- **Source:** Irish Marine Institute or manual entry
|
|
- **Pros:** No API limits, always available
|
|
- **Cons:** Requires manual updates, less accurate
|
|
|
|
**Current Setup:**
|
|
The system currently returns mock tide data for demonstration. To enable real data:
|
|
|
|
1. Sign up for Stormglass.io free tier
|
|
2. Add API key to `.env`:
|
|
```env
|
|
STORMGLASS_API_KEY="your_api_key_here"
|
|
```
|
|
3. Update `externalDataService.ts` to use Stormglass API
|
|
|
|
**Note:** With only 10 free requests per day, consider:
|
|
- Caching aggressively (tide data changes slowly)
|
|
- Using for specific locations only
|
|
- Implementing static tide tables for common beaches
|
|
|
|
---
|
|
|
|
### 3. Caching
|
|
|
|
**Current Implementation:** In-memory caching (no external service required)
|
|
|
|
**How it works:**
|
|
- API responses cached for 6 hours
|
|
- Automatic cache expiration
|
|
- No setup required
|
|
|
|
**Optional: Redis for Production**
|
|
|
|
If you want persistent caching across server restarts:
|
|
|
|
**Docker:**
|
|
```bash
|
|
docker run -d -p 6379:6379 --name redis redis:alpine
|
|
```
|
|
|
|
**Add to `.env`:**
|
|
```env
|
|
REDIS_URL="redis://localhost:6379"
|
|
```
|
|
|
|
Then update `src/config/redis.ts` to use Redis client instead of in-memory cache.
|
|
|
|
**Note:** For most use cases, in-memory caching is sufficient.
|
|
|
|
---
|
|
|
|
## Optional APIs (Placeholder Implementations)
|
|
|
|
These APIs are not fully implemented yet but have placeholder endpoints:
|
|
|
|
### 4. EPA Ireland Water Quality Data
|
|
|
|
**Purpose:** Water quality monitoring data for Irish beaches and coastal areas.
|
|
|
|
**Available Data Sources:**
|
|
|
|
#### Option A: EU Bathing Water Quality (Recommended)
|
|
- **Source:** European Environment Agency
|
|
- **API:** https://www.eea.europa.eu/data-and-maps/data/bathing-water-directive-status-of-bathing-water
|
|
- **Cost:** Free, no API key required
|
|
- **Coverage:** All EU designated bathing waters including Ireland
|
|
- **Data Format:** CSV/XML downloads or REST API
|
|
- **Update Frequency:** Annual with in-season updates
|
|
- **Documentation:** https://www.eea.europa.eu/help/glossary/eea-glossary/bathing-water
|
|
|
|
**How to Access:**
|
|
1. Use EEA's REST API endpoint
|
|
2. Filter by country code "IE" for Ireland
|
|
3. Parse XML/JSON response
|
|
4. No authentication required
|
|
|
|
**Example Endpoint:**
|
|
```
|
|
https://www.eea.europa.eu/api/EMIS/get_bathing_water_data?country=IE
|
|
```
|
|
|
|
#### Option B: EPA Ireland GeoPortal
|
|
- **Source:** EPA Ireland
|
|
- **Portal:** https://gis.epa.ie/EPAMaps/
|
|
- **Cost:** Free, no API key required
|
|
- **Method:** Web scraping or manual data download
|
|
- **Data:** Real-time and historical water quality data
|
|
|
|
**Access Methods:**
|
|
1. **Manual Download:** Download datasets from EPA GeoPortal
|
|
2. **Web Scraping:** Parse EPA maps (requires permission)
|
|
3. **Contact EPA:** Request API access at info@epa.ie
|
|
|
|
#### Option C: Marine Institute Ireland
|
|
- **Source:** Marine Institute
|
|
- **Portal:** https://data.marine.ie/
|
|
- **Cost:** Free, no API key required
|
|
- **Data:** Marine water quality, oceanographic data
|
|
- **Format:** Various formats including API access
|
|
|
|
**How to Access:**
|
|
1. Browse datasets at https://data.marine.ie/
|
|
2. Use ERDDAP server for programmatic access
|
|
3. No API key required for public data
|
|
|
|
#### Option D: Static Database (Simplest)
|
|
- **Method:** Create a database of Irish beaches with latest quality ratings
|
|
- **Source:** Manual entry from EPA/EU reports
|
|
- **Update:** Quarterly or annually
|
|
- **Pros:** Fast, reliable, no external dependencies
|
|
- **Cons:** Requires manual updates
|
|
|
|
**Recommended Approach:**
|
|
Use **EU Bathing Water Quality API** (Option A) as it's:
|
|
- Free and public
|
|
- No API key required
|
|
- Covers all Irish beaches
|
|
- Regularly updated
|
|
- Easy to integrate
|
|
|
|
**Status:** Placeholder implementation - needs integration with one of the above sources
|
|
**API Key:** Not required for any option
|
|
|
|
---
|
|
|
|
### 5. Beach Quality Ratings (Blue Flag, Green Coast)
|
|
|
|
**Purpose:** Beach quality certifications and awards.
|
|
|
|
**Sources:**
|
|
- [Blue Flag Ireland](https://www.blueflagireland.org/)
|
|
- [Green Coast Awards](https://www.greencoastawards.ie/)
|
|
- [An Taisce](https://www.antaisce.org/)
|
|
|
|
**Status:** Placeholder implementation - returns mock data
|
|
**Cost:** Free (public data)
|
|
**API Key:** Not required
|
|
|
|
**To Implement:**
|
|
- These organizations may not have public APIs
|
|
- May require web scraping or manual data entry
|
|
- Could create a static database of certified beaches
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Complete `.env` File Example
|
|
|
|
```env
|
|
# Database
|
|
DATABASE_URL="mysql://beach_litter_user:BeachClean2024!@192.168.4.20:3306/beach_litter_mapping"
|
|
|
|
# Clerk Authentication
|
|
CLERK_SECRET_KEY="sk_test_your_secret_key_here"
|
|
CLERK_WEBHOOK_SECRET="whsec_your_webhook_secret_here"
|
|
|
|
# Server
|
|
PORT=3000
|
|
NODE_ENV=development
|
|
|
|
# File Upload
|
|
UPLOAD_DIR=./uploads
|
|
MAX_FILE_SIZE=5242880
|
|
|
|
# Redis Cache
|
|
REDIS_URL="redis://localhost:6379"
|
|
|
|
# External APIs
|
|
OPENWEATHER_API_KEY="your_openweather_api_key_here"
|
|
WORLDTIDES_API_KEY="your_worldtides_api_key_here"
|
|
```
|
|
|
|
---
|
|
|
|
## Testing the Integration
|
|
|
|
After configuring your API keys, test the integration:
|
|
|
|
```bash
|
|
# Start Redis (if not already running)
|
|
redis-cli ping
|
|
|
|
# Run the test script
|
|
npm run dev
|
|
# In another terminal:
|
|
cd server
|
|
npx tsx src/scripts/testExternalAPIs.ts
|
|
```
|
|
|
|
Or test via HTTP:
|
|
|
|
```bash
|
|
# Test weather endpoint
|
|
curl http://localhost:3005/api/external/weather/53.3498/-6.2603
|
|
|
|
# Test tide endpoint
|
|
curl http://localhost:3005/api/external/tides/53.3498/-6.2603
|
|
|
|
# Test all data
|
|
curl http://localhost:3005/api/external/all/53.3498/-6.2603
|
|
```
|
|
|
|
---
|
|
|
|
## Cache Behavior
|
|
|
|
- **Cache Duration:** 6 hours (21,600 seconds)
|
|
- **Cache Keys:** Based on coordinates rounded to 4 decimal places
|
|
- **Fallback:** If Redis is unavailable, API calls work without caching (degraded mode)
|
|
|
|
**Cache Key Format:**
|
|
```
|
|
weather:53.3498:-6.2603
|
|
tide:53.3498:-6.2603
|
|
water-quality:53.3498:-6.2603
|
|
beach-quality:53.3498:-6.2603
|
|
```
|
|
|
|
---
|
|
|
|
## Cost Optimization Tips
|
|
|
|
1. **Use Redis Caching:** Reduces API calls by 6 hours per location
|
|
2. **Round Coordinates:** Cache keys use 4 decimal places (~11m precision)
|
|
3. **Monitor Usage:** Check API dashboards regularly
|
|
4. **Rate Limiting:** Implement rate limiting on frontend to prevent abuse
|
|
5. **Lazy Loading:** Only fetch data when user views environmental panel
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Weather/Tide Data Returns 503
|
|
|
|
**Cause:** API key not configured or invalid
|
|
|
|
**Solution:**
|
|
1. Check `.env` file has correct API keys
|
|
2. Verify API keys are active in provider dashboards
|
|
3. Check API key hasn't exceeded rate limits
|
|
4. Restart server after adding keys
|
|
|
|
### Redis Connection Errors
|
|
|
|
**Cause:** Redis not running or wrong connection URL
|
|
|
|
**Solution:**
|
|
1. Verify Redis is running: `redis-cli ping`
|
|
2. Check `REDIS_URL` in `.env`
|
|
3. For Docker: ensure container is running
|
|
4. Check firewall isn't blocking port 6379
|
|
|
|
### Data Not Updating
|
|
|
|
**Cause:** Cached data still valid
|
|
|
|
**Solution:**
|
|
1. Wait for cache to expire (6 hours)
|
|
2. Clear cache manually: `redis-cli FLUSHALL`
|
|
3. Use different coordinates to bypass cache
|
|
|
|
---
|
|
|
|
## Production Deployment
|
|
|
|
### Environment Variables
|
|
|
|
Set these in your production environment:
|
|
|
|
```env
|
|
NODE_ENV=production
|
|
REDIS_URL="redis://your-production-redis-url"
|
|
OPENWEATHER_API_KEY="your_production_key"
|
|
WORLDTIDES_API_KEY="your_production_key"
|
|
```
|
|
|
|
### Recommended Services
|
|
|
|
**Redis Hosting:**
|
|
- **Upstash** (Recommended for serverless): https://upstash.com/
|
|
- **Redis Cloud**: https://redis.com/try-free/
|
|
- **Railway**: https://railway.app/
|
|
|
|
**API Key Management:**
|
|
- Store keys in environment variables (never commit to git)
|
|
- Use secrets management (AWS Secrets Manager, etc.)
|
|
- Rotate keys periodically
|
|
- Monitor usage and set up alerts
|
|
|
|
---
|
|
|
|
## Future Enhancements
|
|
|
|
1. **EPA Integration:** Implement real EPA water quality API
|
|
2. **Beach Database:** Create static database of Irish beaches with ratings
|
|
3. **Marine Biodiversity:** Integrate with GBIF or similar databases
|
|
4. **Forecast Data:** Add weather forecasts (3-7 days)
|
|
5. **Historical Data:** Store and display historical trends
|
|
6. **Alert System:** Notify users of poor water quality or weather warnings
|
|
|
|
---
|
|
|
|
## Support
|
|
|
|
- **OpenWeatherMap Support:** https://openweathermap.org/faq
|
|
- **WorldTides Support:** support@worldtides.info
|
|
- **Redis Documentation:** https://redis.io/docs/
|
|
|