Do a mass db.delete on App Engine, without eating CPU

We've got a reasonably-sized database on Google App Engine - just over 50,000 entities - that we want to clear out stale data from. The plan was to write a deferred task to iterate over the entities we no longer wanted, and delete them in batches. One complication is that our entities also have child entities that we also want to purge -- no problem, we thought; we'd just query the da

在App Engine上做一个大量的db.delete,而不用吃CPU

我们在Google App Engine上有一个尺寸合理的数据库 - 只有超过50,000个实体 - 我们希望从中清除陈旧的数据。 计划是编写一个延迟任务来迭代我们不再需要的实体,并批量删除它们。 一个复杂因素是我们的实体也有我们也想清除的子实体 - 我们认为没有问题; 我们只需查询这些实体的数据存储,然后将它们放在父代的同一时间: query = ParentKind.all() query.count(100) query.filter('bar =', 'foo') to_delete = [] for entit

Give me some awesome Web2py Tip & Tricks which makes you more productive

I have been working with web2py following its official web2py book and got to learn too many things. But a book never teaches a productive workflow. I would like to know what workflow you people use to increase you productivity and understanding of the framework as a whole. Ex: I often use Web2py interactive shell to play with database objects and understand how they work. I use Komodo Edit

给我一些令人敬畏的Web2py提示和技巧,让你更高效

我一直在使用web2py,遵循它的官方web2py书,并且学到了太多东西。 但是一本书从不教授高效的工作流程。 我想知道你们用什么工作流程来提高你的工作效率和对整个框架的理解。 例如:我经常使用Web2py交互式shell来与数据库对象一起玩,并了解它们是如何工作的。 我使用Komodo Edit进行开发,并使用Google App Engine进行托管。 为了更好地理解web2py,请查看gluon中的文件/ - 例如tools.py非常简单。

Scrape Facebook in Python

I'm interested in getting the number of friends each of my friends on Facebook has. Apparently the official Facebook API does not allow getting the friends of friends, so I need to get around this (somehwhat sensible) limitation somehow. I tried the following: import sys import urllib, urllib2, cookielib username = 'me@example.com' password = 'mypassword' cj = cookielib.CookieJar() opene

在Python中刮脸Facebook

我很想知道Facebook上我的每个朋友的朋友数量。 显然官方Facebook API不允许朋友的朋友,所以我需要绕过这个(某种合理的)限制。 我尝试了以下内容: import sys import urllib, urllib2, cookielib username = 'me@example.com' password = 'mypassword' cj = cookielib.CookieJar() opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) login_data = urllib.urlencode({'email' : username, 'pass' : passwo

What's the difference between lists and tuples?

What's the difference? What are the advantages / disadvantages of tuples / lists? Apart from tuples being immutable there is also a semantic distinction that should guide their usage. Tuples are heterogeneous data structures (ie, their entries have different meanings), while lists are homogeneous sequences. Tuples have structure, lists have order. Using this distinction makes code mor

列表和元组有什么区别?

有什么不同? 元组/列表有什么优点/缺点? 除了元组是不可变的,还有一个应该引导它们用法的语义区别。 元组是异构数据结构(即它们的条目具有不同的含义),而列表是同类序列。 元组有结构,列表有顺序。 使用这种区别使代码更加明确和易于理解。 一个例子是成对的页面和行号来引用书中的位置,例如: my_location = (42, 11) # page number, line number 然后,您可以将其用作字典中的关键字来存储位置的注释。

How to merge two dictionaries with same key names

This question already has an answer here: Difference between append vs. extend list methods in Python 22 answers 如果您想要一个不会改变原始字典并查看名称冲突的合并副本,则可能需要尝试此解决方案: >>> dict1 = {'a': [1], 'b': [2]} >>> dict2 = {'b': [3], 'c': [4]} >>> import copy >>> import itertools >>> def merge(a, b, conflict): new = copy.deepco

如何合并具有相同键名的两个字典

这个问题在这里已经有了答案: Python中的append和extend列表方法之间的区别22回答 如果您想要一个不会改变原始字典并查看名称冲突的合并副本,则可能需要尝试此解决方案: >>> dict1 = {'a': [1], 'b': [2]} >>> dict2 = {'b': [3], 'c': [4]} >>> import copy >>> import itertools >>> def merge(a, b, conflict): new = copy.deepcopy(a) if conflict: for

twisted cred with http digest authentication

I'm trying to implement twisted cred with HTTP Digest Authentication, and I'm having some difficulty. I was able to get it to work with checkPassword, but I don't want to store my passwords in the clear in the database, obviously. I'm storing the MD5 of username:realm:password in my password database, and I'm calculating it using: from twisted.cred._digest import calcHA1

用http摘要认证扭曲信用

我试图通过HTTP摘要式身份验证来实现扭曲的信誉,并且我遇到了一些困难。 我能够使它与checkPassword一起工作,但我不想将密码明确地存储在数据库中。 我在我的密码数据库中存储用户名:realm:password的MD5,并使用以下方法计算它: from twisted.cred._digest import calcHA1 def calc_ha1(self, password, username=None): if username is None: username = self.avatarId realm = self.digest_factory

Limiting floats to two decimal points

I want a to be rounded to 13.95. >>> a 13.949999999999999 >>> round(a, 2) 13.949999999999999 The round function does not work the way I expected. You are running into the old problem with floating point numbers that all numbers cannot be represented. The command line is just showing you the full floating point form from memory. In floating point your rounded version is the

限制浮点数到两个小数点

我想要a四舍五入到13.95。 >>> a 13.949999999999999 >>> round(a, 2) 13.949999999999999 round函数不能按我预期的方式工作。 您正在用浮点数来解决所有数字都无法表示的旧问题。 命令行仅显示内存中的完整浮点格式。 在浮点数中,您的舍入版本是相同的数字。 由于计算机是二进制的,它们将浮点数存储为整数,然后将其除以2的幂,因此13.95将以与125650429603636838 /(2 ** 53)类似的方式表示。 双

How do I check if a string is a number (float)?

What is the best possible way to check if a string can be represented as a number in Python? The function I currently have right now is: def is_number(s): try: float(s) return True except ValueError: return False Which, not only is ugly and slow, seems clunky. However I haven't found a better method because calling float in the main function is even worse.

如何检查字符串是否是数字(浮点数)?

检查一个字符串是否可以在Python中表示为数字的最佳方法是什么? 我现在有的功能是: def is_number(s): try: float(s) return True except ValueError: return False 这不仅丑陋而且缓慢,看起来笨重。 但是我没有找到更好的方法,因为在主函数中调用float更糟。 这不仅丑陋而且缓慢 我会同时争执。 正则表达式或其他字符串解析会更慢更慢。 我不确定有什么比以上更快。 它调用

Convert bytes to a string?

I'm using this code to get standard output from an external program: >>> from subprocess import * >>> command_stdout = Popen(['ls', '-l'], stdout=PIPE).communicate()[0] The communicate() method returns an array of bytes: >>> command_stdout b'total 0n-rw-rw-r-- 1 thomas thomas 0 Mar 3 07:03 file1n-rw-rw-r-- 1 thomas thomas 0 Mar 3 07:03 file2n' However, I'd

将字节转换为字符串?

我正在使用此代码从外部程序获取标准输出: >>> from subprocess import * >>> command_stdout = Popen(['ls', '-l'], stdout=PIPE).communicate()[0] 通信()方法返回一个字节数组: >>> command_stdout b'total 0n-rw-rw-r-- 1 thomas thomas 0 Mar 3 07:03 file1n-rw-rw-r-- 1 thomas thomas 0 Mar 3 07:03 file2n' 但是,我想用普通的Python字符串处理输出。 这样我可以像这样打印它: &

Proper way to declare custom exceptions in modern Python?

What's the proper way to declare custom exception classes in modern Python? My primary goal is to follow whatever standard other exception classes have, so that (for instance) any extra string I include in the exception is printed out by whatever tool caught the exception. By "modern Python" I mean something that will run in Python 2.5 but be 'correct' for the Python 2.6

在现代Python中声明自定义异常的正确方法是什么?

在现代Python中声明自定义异常类的正确方法是什么? 我的主要目标是遵循其他任何标准的异常类,以便(例如)我包含在异常中的任何额外字符串通过任何工具捕获异常来打印出来。 对于“现代Python”,我的意思是可以在Python 2.5中运行的东西,但是对于Python 2.6和Python 3. *做法是“正确的”。 而通过“自定义”,我的意思是一个Exception对象,它可以包含有关错误原因的额外数据:一个字符串,也许还有一些其他与该异常相关的任