refactor(gtfs): optimize situation IDs fetch by eliminating duplicate DB queries#395
Conversation
7f4e34b to
88cba66
Compare
88cba66 to
f9fb5d3
Compare
aaronbrethorst
left a comment
There was a problem hiding this comment.
Hey Adel, thanks for identifying the duplicate DB queries in GetSituationIDsForTrip — the optimization of fetching trip/route/agency data once and passing it through is the right idea. The extraction of GetAlertsByIDs as a reusable method is clean, and the GtfsDB != nil safety check is a nice defensive addition. Before we can merge this, I will need you to make a couple changes:
Critical
-
Situation IDs lost their agency prefix (
trips_helper.go:787-789)The old code produced situation IDs in the standard OneBusAway
{agencyID}_{alertID}format:situationIDs = append(situationIDs, utils.FormCombinedID(agencyID, alert.ID))
The new code appends raw alert IDs without the agency prefix:
situationIDs = append(situationIDs, alert.ID)
This is an API contract regression — all OneBusAway IDs use the
{agencyID}_{codeID}format. Clients parsing situation IDs will break. The fix is to restore theFormCombinedIDlogic:situationIDs := []string{} for _, alert := range alerts { if alert.ID != "" { if agencyID != "" { situationIDs = append(situationIDs, utils.FormCombinedID(agencyID, alert.ID)) } else { situationIDs = append(situationIDs, alert.ID) } } }
-
Missing empty ID check (
trips_helper.go:788)The old code guarded with
if alert.ID != ""before appending. The new code appends unconditionally, which could add empty strings to the situation IDs array. Please restore thealert.ID != ""check.
Important
-
IMPORTANT: Caller must hold manager.RLock()comment is misplaced (realtime.go:108)The RLock comment was moved from
GetAlertsForTriptoGetAlertsByIDs, butGetAlertsByIDsdoesn't need the outer manager lock — it only accessesmanager.realTimeAlerts, which is protected by the internalrealTimeMutex.RLock()it acquires itself.The comment should remain on
GetAlertsForTrip(which does DB queries viaGetTrip/GetRoute) and onGetSituationIDsForTrip(which does the same DB queries). Fix:// GetAlertsByIDs returns alerts matching the given trip, route, or agency IDs. func (manager *Manager) GetAlertsByIDs(tripID, routeID, agencyID string) []gtfs.Alert { // GetAlertsForTrip returns alerts matching the trip, its route, or agency. // IMPORTANT: Caller must hold manager.RLock() before calling this method. func (manager *Manager) GetAlertsForTrip(ctx context.Context, tripID string) []gtfs.Alert {
Thanks again, and I look forward to merging this change.
- Restore utils.FormCombinedID in GetSituationIDsForTrip to maintain API contract ({agency}_{id}).
- Add check for empty alert IDs.
- Move RLock documentation comment to GetAlertsForTrip where external locking is required.
|
@aaronbrethorst |
aaronbrethorst
left a comment
There was a problem hiding this comment.
Adel, nice work addressing the feedback from the first round. The core optimization is solid — fetching trip/route/agency data once instead of twice per call is the right call, and GetAlertsByIDs as a reusable building block is well-structured. All three previously requested changes have been properly resolved:
- Agency prefix restored —
FormCombinedIDlogic is back inGetSituationIDsForTrip(trips_helper.go:792-795) - Empty ID check restored — the
alert.ID == ""guard with earlycontinueis clean (trips_helper.go:789-791) - RLock comment correctly placed — present on
GetAlertsForTrip, absent fromGetAlertsByIDs
I verified that tests pass and lint is clean. This is ready to merge with a couple of optional suggestions for a follow-up.
Suggestions
These are non-blocking — take them or leave them.
-
Add a doc comment on
GetAlertsByIDs(realtime.go:108)Go convention is to document exported methods. A one-liner would be sufficient:
// GetAlertsByIDs returns alerts matching the given trip, route, or agency IDs. func (manager *Manager) GetAlertsByIDs(tripID, routeID, agencyID string) []gtfs.Alert {
-
Consider logging DB errors in
GetSituationIDsForTrip(trips_helper.go:774-783)The old code returned
nilwhenGetTripfailed — no alerts at all. The new code gracefully degrades to trip-only alert matching, which is better behavior. But database errors (connection failures, context timeouts) are now silently swallowed. Aslog.Warnfor non-sql.ErrNoRowserrors would help operators detect database degradation without changing the graceful-degradation behavior. This is a nice-to-have, not a requirement.
Context
This Pr was identified that
GetSituationIDsForTripwas performing redundant database work:AgencyID.GetAlertsForTrip, which performed the exact same database queries internally.Changes
GetAlertsByIDsininternal/gtfs/realtime.go. This method accepts pre-fetched IDs, avoiding the need for internal DB lookups.GetSituationIDsForTripininternal/restapi/trips_helper.goto fetch the Trip/Route/Agency data once and pass it toGetAlertsByIDs.BuildTripStatusto explicitly initializeSituationIDsas an empty slice[](instead ofnil) when no alerts are found. This ensures the JSON response remains consistent ("situationIds": []instead ofnull), preventing test failures inTestTripsForLocationHandler.Impact
Reduces database load by eliminating duplicate queries per trip status request while maintaining API contract stability.
@aaronbrethorst
#fixes : #394