Skip to content

Conversation

DarkDust
Copy link

In preparation of upcoming SwiftSoup changes, this converts the FANotesPage to non-asynchronous processing but speeds up the processing by caching the parsed queries.

Once the next SwiftSoup version is out this can be further improved:

  • @preconcurrency for the import SwiftSoup can be removed since the Evaluator instances are going to be sendable.
  • The CssSelector.select(eval, node) can be changed to node.select(eval) which is nicer to read.
  • try elementsSelect(Self.titleQuery, baseNode.array()) can be replaced by baseNode.select(Self.titleQuery) and the helper function can be deleted.

We could also wait for the next SwiftSoup version before merging this merge request and do the improvements mentioned above. I've created this merge request mostly as a demonstration so it doesn't get lost.

@DarkDust
Copy link
Author

Let's put that on hold, there are more SwiftSoup changes coming which are related to query parsing which may help improve the performance with less code changes: scinfu/SwiftSoup#339

@Ceylo
Copy link
Owner

Ceylo commented Aug 30, 2025

Since scinfu/SwiftSoup#343 was merged and released, shouldn't I get most of the benefit without any source change, just by updating SwiftSoup?
As I understand the added cache removes the need for explicit evaluator creation and use. Am I correct?

@DarkDust
Copy link
Author

Yes, that's right! You'll get warnings like Sending 'foo' risks causing data races because of parallelMap, though.

I just had a test with 5 notes, measuring how long these two lines take:

let noteNodes = try doc.select(notesQuery)
self.noteHeaders = try await noteNodes.parallelMap { try NoteHeader($0) }

That was about 0.009s. With SwiftSoup 2.11 and map instead of parallelMap it was about 0.005s. Might be worth making your own measurements for more complicated stuff like FANotificationsPage: is the performance with SwiftSoup 2.11 and without async worse than your current version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants