What's the idiomatic way to keep track of previous values in clojure?
(defmethod learn [:SARSA :Module] [learner module]
(let [samples (get learner :dataset)]
(for [seq samples]
(let [laststate (atom 0) lastaction (atom 0) lastreward (atom 0)])
;;Do some stuff
;;Update laststate,lastaction,lastreward
)
))
Im using a for loop to iterate through a sequence but perhaps I should use a regular loop and recur? Would a map/reduce be useful here?
Be careful -- in Clojure, it's better to think of for not as a loop, but as a list comprehension -- it takes a collection and returns a modified/filtered version of that collection.
You can do this more idiomatically (in a more functional programming style) by using loop and recur, something like this:
(defmethod learn [:SARSA Module] [learner module]
(loop [samples (get learner :dataset)
last-state 0
last-action 0
last-reward 0]
(if-let [sample (first samples)]
(recur (next samples) (new-last-state) (new-last-action) (new-last-reward))
[last-state last-action last-reward])))
Each time you iterate through with new values for last-state , last-action and last-reward , the (if-let [sample (first samples)] part determines whether there are any samples left to look at -- if there aren't, that means you're at the end of the list, and (first '()) will return nil , so your results will be returned in whatever form you'd like -- see the last line, where I just returned them as a vector. If there are still samples left, we bind the first one to the symbol sample , which you can use for your updated calculations of last-state , etc., then recur with these updated values and (next samples) , which is everything after the first sample in that list.
EDIT: I would generally try and do things using map/reduce whenever I can, but whenever you are trying to do a complicated looping operation where you're tallying and calculating a handful of different statistics, loop/recur is usually the best way to go.
@DaveYarwood alluded to map / reduce in his answer ; here's how you could implement it:
(defmethod learn [:SARSA Module] [learner module]
(reduce (fn [[state action reward] sample]
;; do some stuff and computes new values for state/action/reward
[new-state new-action new-reward])
[0 0 0]
(get learner :dataset)))