Query response size limit on appengine?

 

Appengine docs mention a 1Mb limit on both entity size and batch get requests (db.get()): http://code.google.com/appengine/docs/python/datastore/overview.html

Is there also a limit on the total size of all entities returned by a query for a single fetch() call?

Example query: 

db.Model.all().fetch(1000)

Update: As of 1.4.0 batch get limits have been removed!

  • Size and quantity limits on datastore batch get/put/delete operations have been removed. Individual entities are still limited to 1 MB, but your app may batch as many entities together for get/put/delete calls as the overall datastore deadline will allow for.

    • Theres no longer a limit on the number of entities that can be returned by a query, but the same entity size limit applies when you are actually retrieving / iterating over the entities. This will only be on a single entity at a time though; it is not a limit on the total size of all entities returned by the query.

    Bottom line: as long as you don't have a single entity that is > 1Mb you should be OK with queries.

    I tried it out on production and you can indeed exceed 1 Mb total for a query. I stopped testing at around 20 Mb total response size. 

    from app import models

# generate 1Mb string
a = 'a'
while len(a) < 1000000:
    a += 'a'

# text is a db.TextProperty()
c = models.Comment(text=a)
c.put()


for c in models.Comment.all().fetch(100):
    print c

    Output: 

    <app.models.Comment object at 0xa98f8a68a482e9f8>
<app.models.Comment object at 0xa98f8a68a482e9b8>
<app.models.Comment object at 0xa98f8a68a482ea78>
<app.models.Comment object at 0xa98f8a68a482ea38>
....

    Yes there is a size limit; the quotas and limits section explicitly states there is a 1 megabyte limit to db API calls.

    You will not be able to db.get(list_of_keys) if the total size of the entities in the batch is over 1 megabyte. Likewise, you will not be able to put a batch if the total size of the entities in the batch is over 1 megabyte.

    The 1,000 entity limit has been removed, but (at present) you will need to ensure the total size of your batches is less than 1 megabyte yourself.

    链接地址: http://www.djcxy.com/p/23214.html

    上一篇: Google App Engine：高效的大型删除（约90000 /天）

    下一篇: 在appengine上查询响应大小限制？