Improve the calculation of distances between objects in queryset

I have next models in my Django project:

class Area(models.Model):
    name = models.CharField(_('name'), max_length=100, unique=True)
    ...

class Zone(models.Model):
    name = models.CharField(verbose_name=_('name'),
                            max_length=100,
                            unique=True)
    area = models.ForeignKey(Area,
                             verbose_name=_('area'),
                             db_index=True)
    polygon = PolygonField(srid=4326,
                           verbose_name=_('Polygon'),)
    ...

The Area is like a city, and the Zone is like a district.

So, I want to cache for every zone what is the order with other zones of its area. Something like this:

def store_zones_by_distance():
    zones = {}
    zone_qs = Zone.objects.all()
    for zone in zone_qs:
        by_distance = Zone.objects.filter(area=zone.area_id).distance(zone.polygon.centroid).order_by('distance').values('id', 'name', ...)
        zones[zone.id] = [z for z in by_distance]
    cache.set("zones_by_distance", zones, timeout=None)

But the problem is that it is not efficient and it is not scalable. We have 382 zones, and this function gets 383 queries to DB, and it is very slow (3.80 seconds in SQL time and 4.20 seconds in global time).

is there any way efficient and scalable to get it. I had thought in something like this:

def store_zones_by_distance():
    zones = {}
    zone_qs = Zone.objects.all()
    for zone in zone_qs.prefetch_related(Prefetch('area__zone_set', queryset=Zone.objects.all().distance(F('polygon__centroid')).order_by('distance'))):
        by_distance = zone.area.zone_set.all().values('id', 'name', ...)
        zones[zone.id] = [z.name for z in by_distance]

This obviously does not work, but something like this, caching in SQL (prefetch related) the zones ordered (area__zone_set).

from Improve the calculation of distances between objects in queryset