Saturday, 29 May 2021

Django : bulk upload with confirmation

Yet another question about the style and the good practices. The code, that I will show, works and do the functionality. But I'd like to know is it ok as solution or may be it's just too ugly?

As the question is a little bit obscure, I will give some points at the end.

So, the use case.

I have a site with the items. There is a functionality to add the item by user. Now I'd like a functionality to add several items via a csv-file.

How should it works?

  1. User go to special upload page.
  2. User choose a csv-file, click upload.
  3. Then he is redirected to the page that show the content of csv-file (as a table).
  4. If it's ok for user, he clicks "yes" (button with "confirm_items_upload" value) and the items from file are added to database (if they are ok).

I saw already examples for bulk upload for django, and they seem pretty clear. But I don't find an example with an intermediary "verify-confirm" page. So how I did it :

  1. in views.py : view for upload csv-file page
def upload_item_csv_file(request):
    if request.method == 'POST':
        form = UploadItemCsvFileForm(request.POST, request.FILES)
        if form.is_valid():
            uploaded_file_name = handle_uploaded_item_csv_file(request.FILES['item_csv_file'])
            request.session['uploaded_file'] = uploaded_file_name
            return redirect('show_upload_csv_item')
    else:
        form = UploadItemCsvFileForm()
    return render(request, 'myapp/item_csv_upload.html', {'form': form})
  1. in utils.py : handle_uploaded_item_csv_file - just save the file and return a file-name
def handle_uploaded_item_csv_file(f):
    now = datetime.now()
    # YY_mm_dd_HH_MM
    dt_string = now.strftime("%Y_%m_%d_%H_%M")
    file_name = os.path.join(settings.MEDIA_ROOT, f"tmp_csv/item_csv_{dt_string}.csv")
    with open(file_name, 'wb+') as destination:
        for chunk in f.chunks():
            destination.write(chunk)

    return f"tmp_csv/item_csv_{dt_string}.csv"
  1. in views.py : view for show_upload_csv_item
@transaction.atomic
def show_uploaded_file(request):
    if 'uploaded_file' in request.session :
        file_name = request.session['uploaded_file']
    else :
        print("Something wrong : raise 404")
        raise Http404
    if not os.path.isfile(os.path.join(settings.MEDIA_ROOT, file_name)):
        print("Something wrong, file does not exist : raise 404")
        raise Http404

    with open(os.path.join(settings.MEDIA_ROOT, file_name)) as csvfile :
        fieldnames = ['serial_number', 'type', 'shipping_date', 'comments']
        csv_reader = csv.DictReader(csvfile, delimiter=';', fieldnames=fieldnames)
        list_items = list(csv_reader)

    if request.POST and ("confirm_items_upload" in request.POST) :
        if request.POST["confirm_items_upload"] == "yes" :
            for cur_item in list_items :
                if not cur_item['shipping_date'] :
                    cur_item.pop('shipping_date', None)

                try :
                    Item.objects.create(**cur_item)
                except IntegrityError :
                    messages.warning(request, f"This Item : {cur_item} - already exists. No items were added." )
            os.remove(os.path.join(settings.MEDIA_ROOT, file_name))
            return redirect('items')
    else :
        return render(request, 'myapp/item_csv_uploaded.html', {'items': list_items})
  1. In forms.py : the form is very obvious, but just to be clear
class UploadItemCsvFileForm(forms.Form):
    item_csv_file = forms.FileField()

Here are the questions/points.

a) Even if obviously it could be better, is this solution is acceptable or not at all ?

b) I pass 'uploaded_file' from one view to another using "request.session" is it a good practice? Is there another way to do it without using GET variables?

c) At first my wish was to avoid to save the csv-file. But I could not figure out how to do it? Reading all the file to request.session seems not a good idea for me. Is there some possibility to upload the file into memory in Django?

d) If I have to use the tmp-file. How should I handle the situation if user abandon upload at the middle (for example, he sees the confirmation page, but does not click "yes" and decide to re-write his file). How to remove the tmp-file?

e) Small additional question : what kind of checks there are in Django about uploaded file? For example, how could I check that the file is at least a text-file? Should I do it?

All others remarks are welcome as well.



from Django : bulk upload with confirmation

No comments:

Post a Comment