Development node last seen

[sameh-farouk]

Description

Include the last_seen field in the Node struct, representing the time when the node sent its last uptime report. Clients can use this to determine node availability based on the reporting interval and their chosen allowable window.

Changes

• Added LastSeen field to the Node struct
• Added timestamp validation with the validateTimestampHint helper function
• Properly updates the node's LastSeen field atomicity when processing uptime reports using a database transaction
• Added MigrateNodeLastSeen function that does the following:
• Updates all nodes with their latest uptime report timestamp
• Updates only nodes that have at least one uptime report and do not have last_seen set.

Related Issues

• support returning node status #44

Checklist

• Tests included
• Build pass
• Documentation
• Code format and docstring

[xmonader]

All times in the system should be UTC and never depend on the location of the server, or that the operations are going to configure the server timezone to be UTC. all input times should be UTC, all output times should be UTC, all calculations/comparisons should be using utc times.

e.g finding if the node is online
1- get the last uptime report time in UTC
2- get the server time in UTC
3- compare both times if within acceptable range then it's online else it's offline

for comparisons/time operations use time package not +/- e.g Add, Sub, Since, .. etc

[xmonader]

Why did we move to that for resources?

[sameh-farouk]

Yes, I noticed that, but it's auto-generated and seems unharmful.
I just rerun swag init -g pkg/server/handlers.go --output docs --parseDependency --parseDepth 2 as I always did to update the swagger files since the node model was changed.
Not sure why that location was changed, but I believe that not caused by the changes in this PR? Maybe something related to moving the repo? Any Idea?

[xmonader]

Also I see we have last_seen field, but where is up field or online field, that was the request in the tickethttps://github.com//issues/44 no?

[sameh-farouk]

Short answer:

Using a “lastSeen” timestamp is a cleaner, more performant approach that minimizes changes to our codebase. It avoids the pitfalls of maintaining and updating a boolean status and empowers clients to define what constitutes an “online” node based on their specific requirements.

Long answer:

Based on theissue the goal is to provide a way to determine if a node is online based on its reports. Instead of maintaining a separate boolean flag for online/offline status, I opted to store a timestamp field (as “lastSeen”).

This design offers several advantages:

1. Simplicity & Minimal Code Changes:
   • With the “lastSeen” timestamp, we only need to update it when a node sends an uptime report.
   • A boolean flag, on the other hand, would require us to update it to “online” when a report is received and then implement additional logic to later invalidate or reset that flag when the node fails to report complicating the codebase unnecessarily, another way (which I think you were referring to) is to update it on-fly with each request, where nodes are quired or listed which requires more processing in multiple handlers.

2. Performance:
   • The timestamp approach reduces overhead. Calculating online status based on whether the “lastSeen” time falls within a 40-minute window is straightforward and efficient on the client side.
   • Maintaining a boolean flag server side would force us to potentially recalculate the status on every request, which could add extra processing, especially when handling a list of nodes.

3. Flexibility for Clients:
   • By exposing the “lastSeen” value, the service provides raw data without making a definitive judgment on a node’s online status.
   • This allows clients to decide on the status based on their own criteria, such as the exact reports interval, preferred time windows, or other contextual factors.
   • Essentially, the service communicates what it knows (the time of the last report), and clients can interpret this information as needed.

However, I can update or revert the changes if they still don’t make sense and adopt any suggested implementation if you find it more suitable.

[xmonader]

Problem is: for convenience of the implementation we pushed the complexity/potential bugs into the clients (we usually have 3: go, ts, and dart)

At least provide a filter to query with status=up instead of storing that flag in the database that will does the comparison against the last 40 mins