Tuesday, 23 May 2023

Nuxt custom error page doesn't load for uncaught errors

I'm trying to use a custom error page in Nuxt (see: https://nuxtjs.org/docs/concepts/views/#error-page).

I've created layouts/error.vue, but this page doesn't load for uncaught exceptions, for example:

<script>
export default {
  computed: {
    component() {
      return () => import(`./non-existent-component.vue`);
    },
  }
}
</script>

When this error occurs server-side, I see the standard production error page. Is there a way to catch this in a generic way so that my custom error page loads in that instance?

enter image description here



from Nuxt custom error page doesn't load for uncaught errors

Monday, 22 May 2023

HuggingFace Evaluate a Fine-tuned Zero-Shot Model

I am finetuning the HuggingFace facebook/bart-large-mnli model to suit my need, I use the following parameters:

training_args = TrainingArguments(
    output_dir=model_directory,      # output directory
    num_train_epochs=30,              # total number of training epochs
    per_device_train_batch_size=1,  # batch size per device during training - 16 - Don't go over 1, it's out of memory
    per_device_eval_batch_size=2,   # batch size for evaluation - 64 - Don't go over 2, it's out of memory
    warmup_steps=500,                 # number of warmup steps for learning rate scheduler - 500
    weight_decay=0.01,               # strength of weight decay
)

model = BartForSequenceClassification.from_pretrained("facebook/bart-large-mnli")

trainer = Trainer(
    model=model,                          # the instantiated 🤗 Transformers model to be trained
    args=training_args,                   # training arguments, defined above
    compute_metrics=compute_metrics,      # a function to compute the metrics
    train_dataset=train_dataset,          # training dataset
    eval_dataset=test_dataset             # evaluation dataset
)

# Train the trainer
trainer.train()

The compute_metrics I use is:

import numpy as np
from datasets import Dataset, load_metric
from transformers import EvalPrediction

def compute_metrics(p: EvalPrediction):
  metric_acc = load_metric("accuracy")
  preds = p.predictions[0] if isinstance(p.predictions, tuple) else p.predictions
  preds = np.argmax(preds, axis=1)
  result = {}
  result["accuracy"] = metric_acc.compute(predictions=preds, references=p.label_ids)["accuracy"]
  return result

But no matter how much train or test data I use, or how many epochs, when I use trainer.evaluate() I get an accuracy of 0.5.

My questions are:

  1. How do I improve it?
  2. How do I implement other metrics for the evaluation? for example F1 score.

I tried changing (adding) the metrics to this:

def compute_metrics(p: EvalPrediction):
  load_accuracy = load_metric("accuracy")
  load_f1 = load_metric("f1")
  preds = p.predictions[0] if isinstance(p.predictions, tuple) else p.predictions
  preds = np.argmax(preds, axis=1)
  result = {}
  result["accuracy"] = load_accuracy.compute(predictions=preds, references=p.label_ids)["accuracy"]
  result["f1"] = load_f1.compute(predictions=preds, references=p.label_ids)["f1"]
  return result

But then I got this error while running trainer.evaluate():

ValueError: pos_label=1 is not a valid label. It should be one of [0, 2]


You can refer to my previous question for more details about my finetuning here



from HuggingFace Evaluate a Fine-tuned Zero-Shot Model

Custom sort function in BigQuery

Is there a way to pass a comparison function to sort by a column in BigQuery? I am fine, if required, using a SQL or JS udf. This would be similar to something like Javascript's localeCompare function, returning a -number, 0, or +number depending on if the first value is less than or equal to the second value.

This would allow doing something like:

const comparisonFunction = (val1, val2) => ...
MyValues.sort(comparisonFunction)

For example, if I have the following data:

with tbl as (
  select "date" val union all 
  select "time" union all 
  select "number"
)
select * from tbl order by comparisonFunction(tbl.val)

And for arguments sake, let's say the comparison function is something like:

const RANK_MAP = {"number": 1, "time": 2, "date": 3}
function comparisonFunction(val1, val2) {
    return RANK_MAP[val1] - RANK_MAP[val2];
}

Note that I want this signature (or something like it). I'm not looking for 'converting this into an inline sql function' with something like:

with tbl as (
  select "date" val union all 
  select "time" union all 
  select "number"
)
select
  tbl.val,
  case tbl.val
    when 'number' then 1
    when 'time' then 2
    when 'date' then 3
  end rank
from tbl
order by rank

How could this be done with a udf function similar to the js approach above?



from Custom sort function in BigQuery