I am building a view in Django , which will send a POST request to chat-gpt API, the problem that I am facing that the response from chat-gpt is taking more than 30 seconds (we are having a long prompts) the idea that I have in mind is:
- The client sends a request to the server.
- The server writes the request to a message queue and returns a message ID to the client.
- Another worker is listening to the message queue. It retrieves the request from the queue, sends it to OpenAI,and then writes the response back to the message queue.
- The client periodically sends requests to the server to ask for the response using the previously received message ID.
- The server responds with "pending" until it finds the response in the message queue, at which point it returns the actual response to the client.
the problem is that I have no idea how to achieve that ... I am using gke for hosting the application, I already have some cronjob using some views as well any idea how to deal with this will be so much appreciated here is an example of the view request:
import openai
from app.forms_prompt import PromptForm
from app.models import ModelName
from django.conf import settings
from django.contrib.auth.decorators import login_required
from django.http import HttpRequest
from django.http import HttpResponse
from django.shortcuts import get_object_or_404
from django.shortcuts import redirect
from django.shortcuts import render
from django.utils.translation import gettext_lazy as _
@login_required
def form_prompt(request: HttpRequest, pk: int) -> HttpResponse:
instance = get_object_or_404(ModelName, pk=pk)
openai.api_key = settings.OPENAI_KEY
form = PromptForm(request.POST or None, instance=setkeyword)
# check if form data is valid
if form.is_valid():
prompt = form.cleaned_data["text"]
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[
{"role": "user", "content": prompt},
],
)
instance.specific_field = response["choices"][0]["message"]["content"]
form.save()
return redirect("view_instance_name", instance.pk)
return render(request, "view_prompt_name.html", context)
any suggestion how to follow the solution, will be very helpful, thank you
from How to bypass the TTL of 30 seconds while waiting response of post request from external server
No comments:
Post a Comment