How is the 60s deadline applied to appengine .run and .fetch retrieval?
The appengine docs state that the default deadline for .run() an .fetch() is 60 seconds. I'd like to understand exactly how that deadline is applied, specifically in the context of a task or cron job, where the total process can run for 10 minutes. Imagine we have 1000s of geese:
geese = Goose.all().run(batch_size=100)
for goose in geese:
goose.cook()
# for sake of example, cooking a goose takes 1 second
If I understand correctly, this will retrieve 100 geese from the datastore, and cook each one in turn. When it gets to the 61st goose, more than 60 seconds will have passed, but this wont matter, because all 100 results were retrieved when .run was called. After the 100th goose, it will attempt to retrieve the next batch of 100 geese, and this will cause a deadline exceeded error. Is that right?
Assuming that is right, then what if we do:
geese = Goose.all().run(batch_size=100, limit=100)
for goose in geese:
goose.cook()
# for sake of example, cooking a goose takes 1 second
I assume this will execute w/oa deadline exceeded error. Is that right? Of course, it will only retrieve the first 100 geese. if we want 101-200 then we'll need to query again, probably using a cursor.
Assuming this is right, what is the difference in this situation between
geese = Goose.all().run(batch_size=100, limit=100)
--- and ---
geese = Goose.all().fetch(batch_size=100, limit=100)
For context: we have quite a lot or run()s, and some fetch()s in our production app and see occasional deadline exceptions. We'd like to robustify them, but are finding the docs a little lacking on exactly how the deadline is applied.
No, you've misunderstood what the deadline is referring to. The docs you link to state:
Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error
What has a deadline here is the remote procedure call that fetches the result. How long you take to iterate over them is completely irrelevant. Certainly your reference to a failure when fetching the second set is incorrect: those are two separate calls, and the time taken to process the results of the first call doesn't affect the second call in any way.
In normal circumstances, the RPC will complete within microseconds, but there might be occasions when it has to wait longer; for example if the records are locked by a transaction.
链接地址: http://www.djcxy.com/p/23220.html上一篇: 使用频繁更改的数据在数据存储上搜索API