I want to use a management command to run a one-time analysis of the buildings in Massachusetts. I have reduced the offending code to an 8 line snippet that demonstrates the problem I encounter. The comments just explain why I want to do this at all. I am running the code below verbatim, in an otherwise-blank management command
zips = ZipCode.objects.filter(state='MA').order_by('id')
for zip in zips.iterator():
buildings = Building.objects.filter(boundary__within=zip.boundary)
important_buildings = []
for building in buildings.iterator():
# Some conditionals would go here
important_buildings.append(building)
# Several types of analysis would be done on important_buildings, here
important_buildings = None
When I run this exact code, I find that memory usage steadily increases with each iteration outer loop (I use print('mem', process.memory_info().rss) to check memory usage).
It seems like the important_buildings list is hogging up memory, even after going out of scope. If I replace important_buildings.append(building) with _ = building.pk, it no longer consumes much memory, but I do need that list for some of the analysis.
So, my question is: How can I force Python to release the list of Django models when it goes out of scope?
from How to force Django models to be released from memory
No comments:
Post a Comment