Error when using fire-rules in parallel

[mauricioszabo]

When trying to run Clara (firing rules) over multiple threads, we get an error

Ofending come (somewhat simplified):

(let [session (-> (mk-session 'events-reports.reports.deferred
                              'events-reports.reports.normalize-data
                              :fact-type-fn :fact))
      res (jdbc/query db "SELECT * FROM my_table WHERE ...")
      location-ids (map :id res)]

  (doseq [id location-ids]
    (async/go
     (println "Emitting for" id)
     (jdbc/with-db-connection [db @smart/db]
        (let [facts (search-facts db id)
              rows (-> session
                       (insert-all facts)
                       (fire-rules)
                       (query detailed-anual-report)
                       (->> (map make-detailed-anual-row)))]
         (println "Saving for" id)
         (save-rows rows))))))

Clojure version: Clojure 1.9.0
JVM version: Eclipse OpenJ9 VM 1.8.0_181-b13
Core.Async version: 0.4.490

Exception:

Exception in thread "async-dispatch-4" java.lang.IllegalArgumentException: find not supported on type: clara.rules.engine.Token$reify__26254
        at clojure.lang.RT.find(RT.java:863)
        at clojure.core$select_keys.invokeStatic(core.clj:1539)
        at clojure.core$select_keys.invoke(core.clj:1532)
        at clara.rules.engine$propagate_items_to_nodes$fn__26782.invoke(engine.cljc:154)
        at clara.rules.platform$group_by_seq$fn__26030.invoke(platform.cljc:39)
        at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
        at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
        at clojure.core.protocols$fn__7852.invokeStatic(protocols.clj:169)
        at clojure.core.protocols$fn__7852.invoke(protocols.clj:124)
        at clojure.core.protocols$fn__7807$G__7802__7816.invoke(protocols.clj:19)
        at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
        at clojure.core.protocols$fn__7835.invokeStatic(protocols.clj:75)
        at clojure.core.protocols$fn__7835.invoke(protocols.clj:75)
        at clojure.core.protocols$fn__7781$G__7776__7794.invoke(protocols.clj:13)
        at clojure.core$reduce.invokeStatic(core.clj:6748)
        at clojure.core$reduce.invoke(core.clj:6730)
        at clara.rules.platform$group_by_seq.invokeStatic(platform.cljc:38)
        at clara.rules.platform$group_by_seq.invoke(platform.cljc:27)
        at clara.rules.engine$propagate_items_to_nodes.invokeStatic(engine.cljc:154)
        at clara.rules.engine$propagate_items_to_nodes.invoke(engine.cljc:147)
        at clara.rules.engine.LocalTransport.send_elements(engine.cljc:176)
        at clara.rules.engine.AlphaNode.alpha_activate(engine.cljc:538)
        at clara.rules.engine$flush_updates$flush_all__26940.invoke(engine.cljc:269)
        at clara.rules.engine$flush_updates.invokeStatic(engine.cljc:277)
        at clara.rules.engine$flush_updates.invoke(engine.cljc:252)
        at clara.rules.engine$fire_rules_STAR_.invokeStatic(engine.cljc:1825)
        at clara.rules.engine$fire_rules_STAR_.invoke(engine.cljc:1720)
        at clara.rules.engine.LocalSession.fire_rules(engine.cljc:1909)
        at clara.rules$fire_rules.invokeStatic(rules.cljc:44)
        at clara.rules$fire_rules.invoke(rules.cljc:29)

[mrrodriguez]

I discussed this some on #clara in clojurians Slack, but this is an odd looking exception that seems like perhaps could happen from clj's keyword callsite optimizations if the clara.rules.engine.Token record type is perhaps compiled/loaded multiple times in a way where both instances of the class (from separate classloaders) are interacting with one another at.

So perhaps

1. the session is compiled with "version 1" of clara.rules.engine.Token,
2. the ns is for some reason loaded again
3. later a runtime instance of "version 2" of clara.rules.engine.Token comes across this compiled callsite

This is just my suspicion so far, putting it here in case it helps track down anything more.