feat: add /api/stats analytics endpoint#66
Open
AdityaAsopa wants to merge 2 commits intoisro:masterfrom
Open
Conversation
The spacecraft_missions data had deeply inconsistent schemas — mass appeared as 'weight', 'lift-off_mass', 'spacecraft_mass', 'mass_at_lift-off' and 5 other variants; dates ranged from 'April 19, 1975' to '22 October 2008' to '26-05-1999' across 15+ formats; KALPANA-1 had mission_life stored in the 'mission' field as '7 Years'; and TES appeared as a duplicate entry. spacecrafts.json had only id+name for 113 records. launchers.json had only id for 81 records. customer_satellites.json mixed 'GERMANY' with 'Germany' and 'UK' with 'UNITED KINGDOM'. This commit introduces scripts/normalize_data.py — an idempotent pipeline that parses all date formats to ISO 8601, extracts numeric mass_kg and power_watts from free-text fields (handling edge cases like '15 Sq.m Solar Array generating 1360W'), classifies orbits (LEO/SSO/GEO/Lunar/Failed), infers mission status from launch date + mission life, and normalizes country names. The scraper was re-run against isro.gov.in and the fresh data is merged with existing records — no data is lost, only enriched. All 5 data files now have consistent, documented schemas. spacecrafts are enriched with launch date, vehicle, orbit type, and status from missions. Launchers are classified into 8 vehicle families. All API endpoints remain backward-compatible — same URLs, same structure, just cleaner data. API handlers: removed unused 'fs' imports, fixed misleading variable names (customer_satellites.js loaded data into a var called 'launchers'), added Content-Type: application/json headers, and sanitized error responses. Root endpoint now returns a JSON directory of all available endpoints.
Returns aggregate statistics computed at runtime from the five data collections: record counts, mission status distribution, orbit type breakdown, mission type breakdown, spacecraft status, customer satellite country distribution and total mass, and launcher vehicle family counts. All values derived directly from the JSON data files — nothing hardcoded. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
/api/statsendpoint that returns aggregate statistics computed at runtime across all five data collections.Response shape
{ "totals": { "spacecrafts": 113, "launchers": 81, "customer_satellites": 75, "centres": 44, "spacecraft_missions": 64 }, "spacecraft_missions": { "by_status": { "active": 3, "decommissioned": 44, "failed": 3, "unknown": 14 }, "by_orbit_type": { "GEO": 22, "SSO": 12, ... }, "by_mission_type": { "Communication": 15, ... } }, "spacecrafts": { "by_status": { ... }, "by_orbit_type": { ... } }, "customer_satellites": { "by_country": { "USA": 10, ... }, "total_mass_kg": 84235.2 }, "launchers": { "by_vehicle_family": { "PSLV": 30, ... } } }Design decisions
"unknown"— consistent with the normalization approach in Normalize data schemas, enrich datasets, and fix API handler quality #65; no silent drops.Notes
This endpoint pairs naturally with the data normalization in PR #65 — the structured fields (
orbit_type,status,mission_type) are what make these distributions meaningful. If #65 merges first, this PR applies cleanly on top.Test plan
GET /api/statsreturns valid JSON withContent-Type: application/jsontotal_mass_kgincustomer_satellitesmatches manual sum ofmass_kgfields