Monday 31 October 2022

Dimension error by using Patch Embedding for video processing

I am working on one of the transformer models that has been proposed for video classification. My input tensor has the shape of [batch=16 ,channels=3 ,frames=16, H=224, W=224] and for applying the patch embedding on the input tensor it uses the following scenario:

patch_dim = in_channels * patch_size ** 2
self.to_patch_embedding = nn.Sequential(
        Rearrange('b t c (h p1) (w p2) -> b t (h w) (p1 p2 c)', p1 = patch_size, p2 = patch_size),
        nn.Linear(patch_dim, dim),     ***** (Root of the error)******
    )

The parameters that I am using are as follows:

 patch_size =16 
 dim = 192
 in_channels = 3

Unfortunately I receive the following error that corresponds to the line that has been shown in the code:

Exception has occured: RuntimeError
mat1 and mat2 shapes cannot be multiplied (9408x4096 and 768x192)

I thought a lot on the reason of the error but I couldn't find out what is the reason. How can I solve the problem?



from Dimension error by using Patch Embedding for video processing

How to add property to PostgreSQL database that is hosted on Heroku with Flask/Python?

I have a database for my website that is hosted on Heroku and uses Flask and Python. The model structure looks like:

class MyDataModel(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    property1 = db.Column(db.String(240), default = "")
    property2 = db.Column(db.String(240), default = "")
    property3 = db.Column(db.String(240), default = "")

When I try to update this model to something with an additional property (property4) shown below, the website doesn't work. Is there a way to add an additional property to a model so that the model still functions properly?

class MyDataModel(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    property1 = db.Column(db.String(240), default = "")
    property2 = db.Column(db.String(240), default = "")
    property3 = db.Column(db.String(240), default = "")
    property4 = db.Column(db.String(240), default = "")

The db is set up like:

db = SQLAlchemy()
app = Flask(__name__)
db.init_app(app)


from How to add property to PostgreSQL database that is hosted on Heroku with Flask/Python?

Webcamjs: Live images are not getting captured properly in safari

I'm developing a monitoring system where we need to take images of our user(under their permission ofc) every x minute and the webcam HTML element wouldn't be visible to the user.

But, sometimes the snapped images are giving old images in Safari. I've created a mock app with webcamjs and I could able to reproduce this in safari only when the webcam node is hidden from the viewport.

For hiding webcam element from viewport, I've used the following style

#webcam{
  position: fixed;
  top: -10000px;
  left: -10000px;
}

Steps to reproduce

Specifications

Browser: Safari version 16.0 OS: MacOS 12.6 WebcamJS: 1.0.26

  • Access this url in Safari. For demo purpose, I've snapped the images every 10 seconds and rendered it into the DOM.
  • You could see repeated images rendered into the DOM

Code

function loadWebcamJS(){
        const webcam = document.getElementById("webcam");
        Webcam.set({
          width: 640,
          height: 480
        });
        Webcam.attach(webcam);
        Webcam.on('load', afterLoad);
      }
      const getTime = (d) =>
          `${d.getHours()}:${d.getMinutes()}:${d.getSeconds()}`;

      function snap(){
        Webcam.snap((dataURI) => {
          const results = document.getElementById('results');
          const date = new Date();
          const time = getTime(date);
          
          results.innerHTML += `
          <div class="image">
            <img src=${dataURI}
              alt="Snapped Image">
            <h4>${time}</h4>
          </div>
          
          `

        })
      }
      loadWebcamJS();
      function afterLoad(){
        
        setInterval(() => {
          snap();
        }, 1000 * 10); //Snap images every 10 seconds
        
      }


from Webcamjs: Live images are not getting captured properly in safari

This version of the application is not configured for billing through Google Play (in live application)

When I tried to purchase a product with the in-app purchase I am receiving an error as mentioned below instead of in-app purchase dialog

This version of the application is not configured for billing through Google Play

Yes, there are lots of answers available online but they all suggested adding this particular user as a licensed tester, I am facing this issue in the live application and I can not add a user as a licensed tester.

I also find a workaround for this issue as mentioned below,

  1. Uninstall the app from the device
  2. I removed the google account from the device on which I am facing the issue
  3. Then add a new google account on the same device
  4. Install the app again from the google play store
  5. And try to purchase a product and it worked
  6. Again I uninstall the app
  7. I remove the new google account from the device and add the old one
  8. Install the app from the play store
  9. And try to purchase a product and its shows the error mentioned above

Yes, adding a new google account fix the issue but my question is why this is not working with my existing google account?



from This version of the application is not configured for billing through Google Play (in live application)

Importing Numpy fails after building from source against amd blis

I'm trying to build a local version of Numpy from source against BLIS (for BLAS and CBLAS) and against OpenBLAS for LAPACK.

I started with building BLIS locally for zen3 with CBLAS enabled, like so:

./configure --enable-threading=openmp --enable-cblas --prefix=$HOME/blis zen3

then ran the tests (which all passed) and ran make install. I made sure all relevant files are in the $HOME/blis library (see attached screenshot).

I also built openBLAS locally, no special configs there.

Afterwards, I modified numpy's site.cfg to configure openBLAS and blis folders' accordingly:

[blis]
libraries = blis
library_dirs = /home/or/blis/lib/
include_dirs = /home/or/blis/include/blis
runtime_library_dirs = /home/or/blis/lib/

[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib

I continued by building and installing numpy with:

NPY_BLAS_ORDER=blis NPY_LAPACK_ORDER=openblas NPY_CBLAS_LIBS= python ./numpy/setup.py build -j 32

Note that NPY_CBLAS_LIBS is empty as numpy's build docs say to do so if CBLAS is included in the BLIS library, which it is.

Then, importing numpy resulting in: Original error was: /home/or/.pyenv/versions/3.9.6/lib/python3.9/site-packages/numpy-1.24.0.dev0+998.g6a5086c9b-py3.9-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm

I'm clueless at this point as I couldn't find anything online about this specific case.

Installing numpy from pip (which comes built with openblas) can be imported successfully.

Update 1: While reading make install logs, I found out that it couldn't find my BLIS library files at the location, even though the files are in the specified path. I also tried to recompile and install BLIS in various paths and reconfigure numpy before compiling it, but got the same result.

When I downloaded a pre-compiled version of BLIS from AMD's website, numpy seems to get it, but this isn't the recommended way to go because I'm missing optimizations for Zen3.



from Importing Numpy fails after building from source against amd blis

Multiple imshow on the same plot, with opacity slider

With Plotly, I'd like to display two imshow on the same page, at the same place, with opacity.

This nearly works:

import plotly.express as px, numpy as np
from skimage import io
img = io.imread('https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Crab_Nebula.jpg/240px-Crab_Nebula.jpg')
fig = px.imshow(img)
x = np.random.random((100, 200))
fig2 = px.imshow(x)
fig.show()
fig2.show()

but it displays the two imshow images in two different tabs.

How to display the two "imshow" on the same plot, with an opacity slider for both layers?

For reference, here is the matplotlib equivalent:

import numpy as np, matplotlib.pyplot as plt, matplotlib.widgets as mpwidgets, scipy.misc
x = scipy.misc.face(gray=False)     # shape (768, 1024, 3)
y = np.random.random((100, 133))    # shape (100, 133)
fig, (ax0, ax1) = plt.subplots(2, 1, gridspec_kw={'height_ratios': [5, 1]})
OPACITY = 0.5
img0 = ax0.imshow(x, cmap="jet")
img1 = ax0.imshow(y, cmap="jet", alpha=OPACITY, extent=img0.get_extent())
slider0 = mpwidgets.Slider(ax=ax1, label='opacity', valmin=0, valmax=1, valinit=OPACITY)
slider0.on_changed(lambda value: img1.set_alpha(value))
plt.show()


from Multiple imshow on the same plot, with opacity slider

Sunday 30 October 2022

How can I translate between client coordinates of a HTML video element and picture coordinates?

Let’s say I have a video element where I want to handle mouse events:

const v = document.querySelector('video');

v.onclick = (ev) => {
  ev.preventDefault();
  console.info(`x=${event.offsetX}, y=${event.offsetY}`);
};
document.querySelector('#play').onclick =
  (ev) => v.paused ? v.play() : v.pause();
document.querySelector('#fit').onchange =
  (ev) => v.style.objectFit = ev.target.value;
<button id="play">play/pause</button>
<label>object-fit: <select id="fit">
  <option>contain</option>
  <option>cover</option>
  <option>fill</option>
  <option>none</option>
  <option>scale-down</option>
</select></label>
<video
  style="width: 80vw; height: 80vh; display: block; margin: 0 auto; background: pink;"
  src="https://archive.org/download/BigBuckBunny_124/Content/big_buck_bunny_720p_surround.mp4"
>
</video>

The event object gives me coordinates relative to the bounding box of the player element (pink). How can I convert between those and coordinates of the actual scaled picture?

I don’t particularly care if overflowing coordinates (those beyond the picture frame) are clipped or extrapolated, I don’t mind rounding errors, and I don’t even care much in which units I get them (percentages or pixels). I would, however, like the solution to be robust to changes to object-fit and object-position CSS properties, and I also want to be able to convert bare coordinates without a mouse event.

How can I perform such a conversion?



from How can I translate between client coordinates of a HTML video element and picture coordinates?

Suppressing option processing after a non-option command-line argument

I am writing a wrapper program which accepts a list of arguments that are going to be passed to another program, and some options that are specific to my wrapper. To avoid confusion regarding which options go where, I want any appearance of a non-option argument to suppress processing further arguments as options, as if a -- argument appeared just before it. In other words, something similar to what xterm -e does.

parse_known_args is obviously wrong, because it will ignore unknown options, and pick up options appearing after a non-option.

What is the best way to accomplish this in Python? I would prefer a solution using argparse over parsing the command line manually.



from Suppressing option processing after a non-option command-line argument

Why React goes Infinite when I set state in function body?

If we set the state with the same value component won't re-render, but it's not applicable when I set the state in the function body.

For example, if I set the same state on the button click and the button clicked, the component does not re-rendering on the button click

function Test1() {
  const [name, setName] = useState("Shiva");
  const onButtonClick = () => {
    console.log("Clicked");
    setName("Shiva");
  };
  console.log("Redering");
  return (
    <div>
      <span>My name is {name}</span>
      <button onClick={onButtonClick}>Click Me</button>
    </div>
  );
}

But, when I set the same state before the return statement React goes infinite renderings

function Test2() {
  const [name, setName] = useState("Shiva");
  // ... come stuff
  setName("Shiva");
  console.log("Rendering");
  return (
    <div>
      <span>My name is {name}</span>
    </div>
  );
}

What actually happening internally?



from Why React goes Infinite when I set state in function body?

Multiple period persistence, vectorization, time series python

I have a DataFrame with daily values and I am building out a forecast using various methods predicting the values for the next two weeks.

As a base, naive, forecast I want to simply say the value today is the best forcast for the next two weeks e.g.:

  • the value on 01-Jan-2012 is 100, then I would like the forecast for 02-Jan-2012 to 15-Jan-2022 to be 100
  • the value on 02-Jan-2012 is 110, then I would like the forecast for 03-Jan-2012 to 16-Jan-2022 to be 110
  • etc

This method can then be compared to the other forecasts to see whether they add value over a naive approach.

To backtest this model how can I do this? I have a few years worth of data in a DataFrame, and I want to do something like below. Reading online, I can only find 1 day persistence help whereby simply using something like df.shift(1) does the job.

Pseudocode:
get the first row from the DataFrame
extract the date from the index
extract the value from the column
propogate forward this value for the next fourteen days
save these forecast dates and forecast values

get the second row from the DataFrame
extract the date from the index
extract the value from the column
propogate forward this value for the next fourteen days
save these forecast dates and forecast values

REPEAT...

However, I've read that iterating over rows is advised against and it is better to use something like pandas apply to 'vectorize' the data but I am not sure how to do this. I was thinking of writing a function to predict the next 14 days then using the apply method to call this function, but not sure how to do so or if this is the best way.

I've also read that numpy is very good for these sorts of problems, but again, am not too familiar.

I've set up a sqlite database so I can store forecasts in there if that helps.



from Multiple period persistence, vectorization, time series python

visualize a two-dimensional point set using Python

I'm new to Python and want to perform a rather simple task. I've got a two-dimensional point set, which is stored as binary data (i.e. (x, y)-coordinates) in a file, which I want to visualize. The output should look as in the picture below.

However, I'm somehow overwhelmed by the amount of google results on this topic. And many of them seem to be for three-dimensional point cloud visualization and/or a massive amount of data points. So, if anyone could point me to a suitable solution for my problem, I would be really thankful.

pointset

EDIT: The point set is contained in a file which is formatted as follows:

0.000000000000000   0.000000000000000
1.000000000000000   1.000000000000000
1
0.020375738732779   0.026169010160356
0.050815740313746   0.023209931647163
0.072530406907906   0.023975230642589

The first data vector is the one in the line below the single "1"; i.e. (0.020375738732779, 0.026169010160356). How do I read this into a vector in python? I can open the file using f = open("pointset file")



from visualize a two-dimensional point set using Python

Saturday 29 October 2022

"Error: Member not found: 'JSON'", when trying to write Firebase Cloud Functions in Dart

I already have Firebase Cloud Functions in JavaScript for an app. But I'm trying to arrange it so that I can write them in Dart, which will be automatically converted into JavaScript, following this instruction: https://pub.dev/documentation/firebase_functions_interop/latest/

When I come to the point of:

dart run build_runner build --output=build

(yes, I changed "pub" to "dart", since "pub" wasn't a defined operable or command, and "dart pub" etc was deprecated, according to the response I got... Could this be the root of the problem?),

I get this error:

Building package executable...
Failed to build build_runner:build_runner:
../../../AppData/Local/Pub/Cache/hosted/pub.dartlang.org/front_end-0.1.4+2/lib/src/fasta/crash.dart:96:45: Error: Member not found: 'JSON'.
        ..headers.contentType = ContentType.JSON
                                            ^^^^

and no index.dart.js file is created.

This is what my pubspec.yaml looks like:

name: functions
version: 1.0.0

environment:
  sdk: '>=2.0.0-dev <3.0.0'
#  sdk: '>=2.12.0 <3.0.0'
#  sdk: '>=2.8.0 <3.0.0'

dependencies:
  firebase_functions_interop: 1.0.0
#  firebase_functions_interop: ^1.0.2

dev_dependencies:
  build_runner: 1.0.0
#  build_runner: ^1.9.9
#  build_runner: ^2.0.3
#  build_runner: ^1.9.0
  build_node_compilers: 0.2.0
#  build_node_compilers: ^0.3.1

I have tried to upgrade everything, but since build_node_compilers seems to be quite an abandonded package (last update was 23 months ago), I can't use the latest version of build_runner...

I've done a number of flutter clean, flutter pub get (turned out to be a bad idea, since this is a Dart project), dart pub get, dart pub upgrade, dart pub upgrade --major-versions, firebase @latest etc... but this error won't go away.

Please help! I don't know where to even start looking...



from "Error: Member not found: 'JSON'", when trying to write Firebase Cloud Functions in Dart

Unable to change the StatusBar when at the splashscreen for light and dark themes

I am trying to change the color/style of the StatusBar based on the device being set to light or dark theme, but not having much luck.

The following XAML works once the app has loaded, but does not work on the splash screen:

<ContentPage.Behaviors>
    <toolkit:StatusBarBehavior StatusBarColor="#F8F9FB" StatusBarStyle="DarkContent"  />
</ContentPage.Behaviors>

Look of StatusBar during splashscreen: enter image description here

Look of StatusBar once app has loaded: enter image description here

As you can see, the XAML above does not seem to affect the StatusBar during the splashscreen, as it is still showing it's default purple color with white text. Any idea how to change the StatusBar while the app is showing the splashscreen?

The end goal is the set the statusbar color and icon colors accordingly based on the device being set to light or dark theme. For example, if the device is set to use the dark theme, the statusbar while showing the splashscreen and the app shell should be a dark background with light text/icons. When the device is set to use the light theme, the statusbar while showing the splashscreen and the app shell should be a light background with dark text/icons.

I have checked many many existing questions about this, but they all seem to be obsolete, as I am using API 33.



from Unable to change the StatusBar when at the splashscreen for light and dark themes

Websocket : Get information from Web socket server

I am very new to websockets and trying to create an omegle as an example. A centralized chat server to which every client can connect through a websocket and the the server matches people based on interests. The people can chat through the websocket connection thereafter.

I am just curious about one thing: Once my site goes live and different clients keep connecting, essentially in the background they are connecting to my central server through the websocket. Can't any client run javascript on its chrome console and inject a malicious script or get access to the clients connected to the server already since the connection has been established and its a stateful connection? I am not sure if there is a way to do that. And if there is, what security mechanisms i need to take care of?



from Websocket : Get information from Web socket server

fabricjs how to see if object src is raster or vectorized

I am creating a function, that, if the object image source is vectorized I will make it larger by scaling it up. But if the image is raster I will use a machine learning model "super resolution" to scale it up.

Currently in the code below I am using toDataURL to return a rasterized version of the image every time, however, to achieve the goal above, I would like to know how to first identify the "type" of the object src, if it is raster or vectorized.

const stringifiedObjectJson = JSON.stringify(obj), fabric.util.enlivenObjects([JSON.parse(json)], function (objects) {
  objects.forEach(function (o) {
    o.top -= bound.top;
    o.left -= bound.left;
    canvas.add(o);
  });
  canvas.renderAll();
  return canvas.toDataURL("image/png");
});


from fabricjs how to see if object src is raster or vectorized

A Python script works fine on Windows but throws an error on Mac

I wish to get different product information from this webpage using requests module. I've created a script in Python to get a JSON response by issuing post requests with appropriate parameters.

The script works fine on windows but throws this error JSONDecodeError: Expecting value: line 1 column 1 (char 0) on mac.

Here is how I've tried:

import requests

start_url = 'https://www.jumbo.com/producten/'
link = 'https://www.jumbo.com/api/graphql'

payload = {"operation":"searchResult","variables":{"searchTerms":"","sortOption":"","showMoreIds":"","offSet":0,"pageSize":24,"categoryUrl":"/producten/"},"query":"\n  fragment productFields on Product {\n    id: sku\n    brand\n    badgeDescription\n    category\n    subtitle: packSizeDisplay\n    title\n    image\n    inAssortment\n    availability {\n      availability\n      isAvailable\n      label\n    }\n    isAvailable\n    isSponsored\n    link\n    status\n    retailSet\n    prices: price {\n      price\n      promoPrice\n      pricePerUnit {\n        price\n        unit\n      }\n    }\n    crossSellSkus\n    quantityDetails {\n      maxAmount\n      minAmount\n      stepAmount\n      defaultAmount\n      unit\n    }\n    quantityOptions {\n      maxAmount\n      minAmount\n      stepAmount\n      unit\n    }\n    primaryBadge: primaryBadges {\n      alt\n      image\n    }\n    secondaryBadges {\n      alt\n      image\n    }\n    promotions {\n      id\n      group\n      isKiesAndMix\n      image\n      tags {\n        text\n        inverse\n      }\n      start {\n        dayShort\n        date\n        monthShort\n      }\n      end {\n        dayShort\n        date\n        monthShort\n      }\n      attachments{\n        type\n        path\n      }\n    }\n  }\n\n  query searchResult(\n    $searchTerms: String\n    $filters: String\n    $offSet: Int\n    $showMoreIds: String\n    $sortOption: String\n    $pageSize: Int\n    $categoryUrl: String\n  ) {\n    searchResult(\n      searchTerms: $searchTerms\n      filters: $filters\n      offSet: $offSet\n      showMoreIds: $showMoreIds\n      sortOption: $sortOption\n      pageSize: $pageSize\n      categoryUrl: $categoryUrl\n    ) {\n      canonicalRelativePath\n      categoryIdPath\n      categoryTiles {\n        id\n        label\n        imageLink\n        navigationState\n        siteRootPath\n      }\n      urlState\n      newUrl\n      redirectUrl\n      shelfDescription\n      removeAllAction\n      powerFilters {\n        displayName\n        navigationState\n        siteRootPath\n      }\n      metaData {\n        title\n        description\n      }\n      headerContent {\n        headerText\n        count\n      }\n      helperText {\n        show\n        shortBody\n        longBody\n        header\n        linkText\n        targetUrl\n        messageType\n      }\n      recipeLink {\n        linkText\n        targetUrl\n        textIsRich\n      }\n      guidedNavigation {\n        ancestors {\n          label\n        }\n        displayName\n        dimensionName\n        groupName\n        name\n        multiSelect\n        moreLink {\n          label\n          navigationState\n        }\n        lessLink {\n          label\n          navigationState\n        }\n        refinements {\n          label\n          count\n          multiSelect\n          navigationState\n          siteRootPath\n        }\n      }\n      selectedRefinements {\n        refinementCrumbs {\n          label\n          count\n          multiSelect\n          dimensionName\n          ancestors {\n            label\n            navigationState\n          }\n          removeAction {\n            navigationState\n          }\n        }\n        searchCrumbs {\n         terms\n         removeAction {\n          navigationState\n         }\n        }\n        removeAllAction {\n         navigationState\n        }\n      }\n      socialLists {\n        title\n        totalNumRecs\n        lists {\n          id\n          title\n          followers\n          productImages\n          thumbnail\n          author\n          labels\n          isAuthorVerified\n        }\n      }\n      mainContent {\n        searchWarning\n        searchAdjustments {\n          originalTerms\n          adjustedSearches {\n            key\n            terms {\n              autoPhrased\n              adjustedTerms\n              spellCorrected\n            }\n          }\n        }\n      }\n      productsResultList {\n        pagingActionTemplate {\n          navigationState\n        }\n        lastRecNum\n        totalNumRecs\n        sortOptions {\n          navigationState\n          label\n          selected\n        }\n        products {\n          ...productFields\n          retailSetProducts {\n            ...productFields\n          }\n        }\n      }\n    }\n  }\n"}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36',
    'accept': 'application/json, text/plain, */*',
    'referer': 'https://www.jumbo.com/producten/',
    'origin': 'https://www.jumbo.com',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9,bn;q=0.8',
}

with requests.Session() as s:
    s.headers.update(headers)
    s.get(start_url)
    res = s.post(link,json=payload)
    print(res.json())

How can I make it work on the Mac?

This video demo represents how the script performs when I execute it on windows.



from A Python script works fine on Windows but throws an error on Mac

I cant get rollup to compile properly when using dynamic imports and crypto-js in my svelte web app

I recently tried to codesplit my svelte web app for each page, but I haven't been able to get it to work while using the crypto-js package. If i remove the package everything works. The js compiles with the line import "crypto", and that causes the browser to error and not to work.

rollup.config.js

import resolve from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import livereload from 'rollup-plugin-livereload';
import json from '@rollup/plugin-json';
import { config } from "dotenv"
import replace from '@rollup/plugin-replace';
import { terser } from 'rollup-plugin-terser';

const production = !process.env.ROLLUP_WATCH;

function serve() {
    let server;
    
    function toExit() {
        if (server) server.kill(0);
    }

    return {
        writeBundle() {
            if (server) return;
            server = require('child_process').spawn('npm', ['run', 'start', '--', '--dev'], {
                stdio: ['ignore', 'inherit', 'inherit'],
                shell: true
            });

            process.on('SIGTERM', toExit);
            process.on('exit', toExit);
        }
    };
}
let envVar = {}
Object.entries(config().parsed).map(([prop, value]) => {
    if (prop.startsWith("APP")) envVar[prop] = value
});

export default {
    input: 'src/main.js',
    output: {
        sourcemap: !production,
        format: 'esm',
        name: 'app',
        dir: 'public/build',
    },
    plugins: [
        json(),
        replace({
            __myapp: JSON.stringify({
                env: envVar
            }),
        }),
        svelte({
            // enable run-time checks when not in production
            dev: !production,
            // we'll extract any component CSS out into
            // a separate file - better for performance
            css: css => {
                css.write('bundle.css');
            }
        }),
        // If you have external dependencies installed from
        // npm, you'll most likely need these plugins. In
        // some cases you'll need additional configuration -
        // consult the documentation for details:
        // https://github.com/rollup/plugins/tree/master/packages/commonjs
        commonjs({
            transformMixedEsModules:true,
            sourceMap: !production,
        }),
        resolve({
            browser: true,
            dedupe: ['svelte'],
            preferBuiltins: false
        }),

        // In dev mode, call `npm run start` once
        // the bundle has been generated
        !production && serve(),

        // Watch the `public` directory and refresh the
        // browser on changes when not in production
        !production && livereload('public'),

        // If we're building for production (npm run build
        // instead of npm run dev), minify
        production && terser()
    ],
    watch: {
        clearScreen: false
    }
};

Browser Error Uncaught TypeError: Failed to resolve module specifier "crypto". Relative references must start with either "/", "./", or "../".

main.js

(function(l, r) { if (l.getElementById('livereloadscript')) return; r = l.createElement('script'); r.async = 1; r.src = '//' + (window.location.host || 'localhost').split(':')[0] + ':35729/livereload.js?snipver=1'; r.id = 'livereloadscript'; l.getElementsByTagName('head')[0].appendChild(r) })(window.document);
export { ao as default } from './main-d2838af7.js';
import 'crypto';
//# sourceMappingURL=main.js.map


from I cant get rollup to compile properly when using dynamic imports and crypto-js in my svelte web app

Add df under other df Pandas

I'm using a for to generate a excel file to graph the data from a df so I'm using value_counts but I would like to add under this df a second one with the same data but with percentages so my code is this one:

li = []

for i in range(0, len(df.columns)):
    value_counts = df.iloc[:, i].value_counts().to_frame().reset_index()
    value_percentage = df.iloc[:, i].value_counts(normalize=True).to_frame().reset_index()#.drop(columns='index')
    value_percentage = (value_percentage*100).astype(str)+'%'
    li.append(value_counts)
    li.append(value_percentage)
data = pd.concat(li, axis=1)
data.to_excel("resultdf.xlsx") #index cleaned

Basically I need it to look like this:

enter image description here



from Add df under other df Pandas

Friday 28 October 2022

Android 13 - How to request WRITE_EXTERNAL_STORAGE

I am targeting my Android app for Android 13 (API 33)

The WRITE_EXTERNAL_STORAGE permission seems to be working fine below API 33 i.e. Android 12 and less but the runtime permission popup for WRITE_EXTERNAL_STORAGE won't appear when running the app on Android 13.

My Android app creates one keystore file in app's private storage.

The behaviour changes for Android 13 mention this:

If your app targets Android 13, you must request one or more new permissions instead of the READ_EXTERNAL_STORAGE.

The new permissions are:

  • Images and photos: READ_MEDIA_IMAGES
  • Videos: READ_MEDIA_VIDEO Audio
  • Audio files: READ_MEDIA_AUDIO

I didn't find any information about this in the official documentation. The documentation is focusing on media files only without any word about other file types.

https://developer.android.com/about/versions/13/behavior-changes-13#granular-media-permissions



from Android 13 - How to request WRITE_EXTERNAL_STORAGE

OpenVPN Python Scanner

I'm trying to create a script in Python that can scan an OpenVPN port over TCP, my general idea is to try and connect to the server by doing a TCP handshake and then sending it to the server

P_CONTROL_HARD_RESET_CLIENT_V2

and if the server answers me - so it's an OpenVPN port. This is my code:

import socket
  

target = IP
port = PORT
  
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.setdefaulttimeout(1)
s.connect((target, port))
print("Connected")
s.send() # P_CONTROL_HARD_RESET_CLIENT_V2
data = s.recv(1024)
print(data)

My main issue is that I don't know how to send P_CONTROL_HARD_RESET_CLIENT_V2, I searched how it should look and couldn't find it.

I tried to use:

senddata= "\x38\x01\x00\x00\x00\x00\x00\x00\x00"
s.send(senddata.encode())
result = s.recv(1024)
print(result)

like in here: https://serverfault.com/questions/262474/how-to-check-that-an-openvpn-server-is-listening-on-a-remote-port-without-using but it didn't work and the answer from the server was b''.

I was able to make this thing:

senddata = "\x00\x36\x38\xe2\x9f\x68\xf2\x3d\x76\x15\x2e\x00\x00\x00\x01\x63\x5a\x3f\x14\x10\xf1\xb9\xd3\x98\xb5\x36\xb9\xbd\x48\x70\xaa\xc7\x29\x2c\x4c\x98\xd0\x17\xdb\x3b\x42\xf0\xa9\x4e\xbd\x65\xbd\x2f\x12\x37\xf2\x10\xb8\x95\xc6\x0a"
s.send(senddata.encode())
result = s.recv(1024)
print(result)

and this time when I used Wireshark I saw that the packet is P_CONTROL_HARD_RESET_CLIENT_V2 (malformed) but I didn't get a response.



from OpenVPN Python Scanner

Nuxt proxy works in dev mode but fails in generated/staging, why?

To avoid users know what endpoint we are requesting data, we are using @nuxtjs/proxy

This config in nuxt.config.js

const deployTarget = process.env.NUXTJS_DEPLOY_TARGET || 'server'
const deploySSR = (process.env.NUXTJS_SSR === 'true') || (process.env.NUXTJS_SSR === true)

And the proxy settings

proxy: {
  '/api/**/**': {
    changeOrigin: true,
    target: process.env.VUE_APP_API_URL,
    secure: true,
    ws: false,
    pathRewrite: { '^/api/': '' }
  }
},

Also we deploy like so

NUXTJS_DEPLOY_TARGET=server NUXTJS_SSR=false nuxt build && NUXTJS_DEPLOY_TARGET=server NUXTJS_SSR=false nuxt start

Also in the httpClient that normally is

constructor (basePath, defaultTimeout, fetch, AbortController) {
  this.basePath = basePath
  this.defaultTimeout = parseInt(defaultTimeout, 10) || 1000
  this.isLocalhost = !this.basePath || this.basePath.includes('localhost')
  this.fetch = fetch
  this.AbortController = AbortController
}

Has been modified like so:

constructor (basePath, defaultTimeout, fetch, AbortController) {
  this.basePath = '/api'
  this.defaultTimeout = parseInt(defaultTimeout, 10) || 1000
  this.isLocalhost = !this.basePath || this.basePath.includes('localhost')
  this.fetch = fetch
  this.AbortController = AbortController
}

The fetch options are

_getOpts (method, options) {
  const opts = Object.assign({}, options)
  opts.method = opts.method || method
  opts.cache = opts.cache || 'no-cache'
  opts.redirect = opts.redirect || 'follow'
  opts.referrerPolicy = opts.referrerPolicy || 'no-referrer'
  opts.credentials = opts.credentials || 'same-origin'
  opts.headers = opts.headers || {}
  opts.headers['Content-Type'] = opts.headers['Content-Type'] || 'application/json'
  if (typeof (opts.timeout) === 'undefined') {
    opts.timeout = this.defaultTimeout
  }

  return opts
}

So that's making a request to https://api.anothersite.com/api/?request..

And in localhost using npm run dev its working just fine, it requests and fetchs the desired data.

But some how, when we deploy it to the staging environment all those request return

{ "code": 401, "data": "{'statusCode':401,'error':'Unauthorized','message':'Invalid token.'}", "json": { "statusCode": 401, "error": "Unauthorized", "message": "Invalid token." }, "_isJSON": true }

Note that

  • the front is being deployed to ourdomain.com that has basic auth and we are properly authenticated
  • The requests in both in local and staging are done to api.anothersite.com without basic auth where the data is served from a Strapi (it doesn't need any token)

is it posible that this response that we are getting is because of the basic auth from the frontend? even that we are logged in?



from Nuxt proxy works in dev mode but fails in generated/staging, why?

Show names of positional arguments inlayed

I've recently been doing some programming in Rust using Clion. This provides a utility whereby when you pass positional arguments to a function it will show you which argument will pick up each value (as if you had passed them in as keyword arguments) see below:

edit with parameter names inlayed

I've found this to be a very nice feature as it's made code a fair bit more readable and prevented me making a couple of stupid mistakes. Therefore, I'd love to have this functionality in PyCharm for programming Python, does anyone know of a plugin which can do this?



from Show names of positional arguments inlayed

Shiny renderUI does not show different D3 Plots after Changing the Tab in TabsetPanel

What I want to do

I am trying to build a shiny app, that shows different interactive HTML-plots using a tabset panel. They all should show up in the same call of renderUI. As shown below in my example, my plots are based on the d3 framework (as in the packages networkD3 andprocessanimateR).

The Problem

Unfortunately, only the first plot that is generated is shown by renderUI. Once the tab is changed, the second plot does not show up. processanimateR gives me the error Failed to render the graph. It is probably too large. Original error: TypeError: d3.create is not a function at PATokens.insertTokens (<anonymous>:185:25) at <anonymous>:52:33 at <anonymous>:105:9. Of course, the plot is not too large to be shown (data-wise), as changing the tab-sequence generates it perfectly. This behavior is independent of which plot shows first. In the example below, I marked the lines you can use to swap tab01 and tab02 to see what happens.

What I tried

By coming up with the smallest example I could, I possibly ruled out a lot of sources for error. Additionally, I have tried using multiple uiOutput (one per tab) without success. The problem persists. As this minimal example suggests, it might be a problem with d3 running in the background. Maybe there's some graphical engine that doesn't close on tab-change or something else js related.

My question

Is there a way to show multiple d3 plots in a shiny app, making them all show up as intended?

Thank you in advance!

My example

#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# Settings
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# Packages
sapply(c("shiny","processanimateR","bupaR","networkD3"),require,character.only=T)

# data
x <- structure(list(
    activity = c(
        "action 1", "action 2", "action 3", 
        "action 5", "action 2", "action 3", "action 25", "action 26", 
        "action 2", "action 3", "action 1", "action 2", "action 3", "action 1", 
        "action 2", "action 3", "action 1", "action 2", "action 3", "action 5", 
        "action 2", "action 3", "action 25", "action 26", "action 2", 
        "action 3", "action 1", "action 2", "action 3", "action 1", "action 2", 
        "action 3"),
    id = c(494.18, 494.18, 494.18, 485.56, 485.56, 485.56, 
           413.99, 413.99, 413.99, 413.99, 439.49, 439.49, 439.49, 466.38, 
           466.38, 466.38, 494.18, 494.18, 494.18, 485.56, 485.56, 485.56, 
           413.99, 413.99, 413.99, 413.99, 439.49, 439.49, 439.49, 466.38, 
           466.38, 466.38), .order = 1:32,
    activity_instance_id_by_bupar = c(1L, 
                                      2L, 3L, 16L, 17L, 18L, 50L, 51L, 52L, 53L, 60L, 61L, 62L, 71L, 
                                      72L, 73L, 1L, 2L, 3L, 16L, 17L, 18L, 50L, 51L, 52L, 53L, 60L, 
                                      61L, 62L, 71L, 72L, 73L),
    lifecycle_id = c("start", "start", 
                     "start", "start", "start", "start", "start", "start", "start", 
                     "start", "start", "start", "start", "start", "start", "start", 
                     "complete", "complete", "complete", "complete", "complete", "complete", 
                     "complete", "complete", "complete", "complete", "complete", "complete", 
                     "complete", "complete", "complete", "complete"),
    timestamp = structure(c(1424304001, 
                            1377475201, 1516579201, 1384128001, 1375401601, 1397952001, 1328486401, 
                            1364688001, 1318032001, 1384905601, 1348099201, 1342483201, 1366416001, 
                            1361404801, 1358726401, 1384905601, 1425168001, 1378339201, 1517443201, 
                            1384992001, 1376265601, 1398816001, 1329350401, 1365552001, 1318896001, 
                            1385769601, 1348963201, 1343347201, 1367280001, 1362268801, 1359590401, 
                            1385769601),
                          tzone = "UTC", class = c("POSIXct", "POSIXt")), 
    trace = c("action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 5,action 3", 
              "action 2,action 5,action 3", "action 2,action 5,action 3", 
              "action 2,action 25,action 26,action 3", "action 2,action 25,action 26,action 3", 
              "action 2,action 25,action 26,action 3", "action 2,action 25,action 26,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 5,action 3", 
              "action 2,action 5,action 3", "action 2,action 5,action 3", 
              "action 2,action 25,action 26,action 3", "action 2,action 25,action 26,action 3", 
              "action 2,action 25,action 26,action 3", "action 2,action 25,action 26,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3", 
              "action 2,action 1,action 3", "action 2,action 1,action 3"
    ),
    n = c(3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 3, 
          3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3),
    absolute_frequency = c(3L, 
                           3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 
                           3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 
                           3L),
    relative_frequency = c(0.272727272727273, 0.272727272727273, 
                           0.272727272727273, 0.0909090909090909, 0.0909090909090909, 
                           0.0909090909090909, 0.0909090909090909, 0.0909090909090909, 
                           0.0909090909090909, 0.0909090909090909, 0.272727272727273, 
                           0.272727272727273, 0.272727272727273, 0.272727272727273, 
                           0.272727272727273, 0.272727272727273, 0.272727272727273, 
                           0.272727272727273, 0.272727272727273, 0.0909090909090909, 
                           0.0909090909090909, 0.0909090909090909, 0.0909090909090909, 
                           0.0909090909090909, 0.0909090909090909, 0.0909090909090909, 
                           0.272727272727273, 0.272727272727273, 0.272727272727273, 
                           0.272727272727273, 0.272727272727273, 0.272727272727273)),
    row.names = c(NA, 
                  -32L),
    class = c("eventlog", "log", "tbl_df", "tbl", "data.frame"
    ),
    case_id = "id", activity_id = "activity", activity_instance_id = "activity_instance_id_by_bupar",
    lifecycle_id = "lifecycle_id",
    resource_id = "id",
    timestamp = "timestamp")
y <- jsonlite::fromJSON(paste0('https://cdn.rawgit.com/christophergandrud/networkD3/','master/JSONdata/energy.json'))

#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# Shiny-App UI ----
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ui <- shinyUI(fluidPage(
    title="shiny example",
    sidebarLayout(
        position="left",
        sidebarPanel(),
        mainPanel(
            tags$div(
                do.call(
                    tabsetPanel,
                    c(id='tabs',
                      lapply(1:2,function(y){
                          tabPanel(id=paste0("tab0",y),value=paste0("tab0",y),title=strong(paste0("This is tab0",y),align="left",style="black"),selected=1)
                      })
                    )
                ),
                uiOutput("out_ui")
            )
        )
    ),
    style=paste0("font-family: Arial;")
))

#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# Shiny-App Server ----
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
server <- function(input,output,session){
    output$out_ui <- renderUI({
        
        # X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
        # >>>> Problem Segment <<<< ----
        # X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
        # Try: Chance this to "tab01"
        if(input$tabs=="tab02"){
            
            # Process
            animate_process(
                x,
                processmap=process_map(
                    x,
                    type=frequency("absolute",color_scale="Paired",color_edges="Greys"),
                    rankdir="TB",render=F,fixed_edge_with=T,
                    layout=layout_pm(
                        edge_weight=F,
                        edge_cutoff=0)),
                mapping=token_aes(
                    size=token_scale(8),
                    color=token_scale(
                        range="black",
                        scale="ordinal"
                    ),
                    opacity=token_scale(100)
                ),
                duration=60,mode="absolute",timeline=T,jitter=10,initial_time=0,
                repeat_count=10,repeat_delay=2,width=1200,height=900,
                sizingPolicy=htmlwidgets::sizingPolicy(
                    browser.fill=T,viewer.fill=F,
                    knitr.figure=F,knitr.defaultWidth="100%",knitr.defaultHeight="1000")
            )
            
        # X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
        # >>>> Problem Segment <<<< ----
        # X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X
        # Try: Chance this to "tab02"
        }else if(input$tabs=="tab01"){
            
            # Sankey
            sankeyNetwork(
                Links=y$links,Nodes=y$nodes,Source='source',
                Target='target',Value='value',NodeID='name',
                units='TWh',fontSize=12,nodeWidth=30
            )
        }
    })
}
shinyApp(ui=ui,server=server)

#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
#XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


from Shiny renderUI does not show different D3 Plots after Changing the Tab in TabsetPanel

Thursday 27 October 2022

No symbols loaded for c++ in mixed debugging (from Python)

I have a large project where the major part computation heavy stuff is written in c++ and the "glue code" and the start script is written in Python. The code has been compiled with the VS 2017 compiler (V141) and Python 3.7.0-32bit for a few years now and I want to upgrade to V143 and Python 3.10-64, I can build and run the code with the new environment. I get some small errors when I run the test suite (probably due to the change from 32bit to 64bit.

The issue occurs when I try to debug the C++ code in Visual Studio 2022. I can debug the Python code without any errors but when I want to debug the code I can't get any symbols from the c++ to load.

I've tried to follow this guide: https://learn.microsoft.com/en-us/visualstudio/python/debugging-mixed-mode-c-cpp-python-in-visual-studio?view=vs-2022 (But I fail to enable both Native and Python in step 2 of Enable mixed-mode debugging in a Python project. When I select Python in Select Code type I get this error: "Python debugging is not compatible with Native. Would you like to uncheck Native?"

Similar I don't find the option Python/Native debugging in step 2 of "Enable mixed-mode debugging in a C/C++ project" and I've installed Python native development tools using the VS installer etc.)

I've built the code with debug info etc. and enabeled Native debugging in the Python project in VS 2022 and when I run the code with debugging the break points in the c++ doesn't get hit. (the pdb, pyd, lib and exp files should have the extension project_name.cp310-win_amd64.pdb for python 3.10-64 right?)

I can also see that it doesn't load the Symbols for my 4 c++ projects (Screenshot of the loaded modules after pausing the debugging while running the code): Print screen when paused debugging with python 3.10-64

Compared to the list of modules when I run the project with Python 3.7.0:

Print screen when paused debugging with python 3.7.0-32

I've tried to run the project in vscode but I can't get it to load the debugging there either. Any suggestion how to get the symbols loaded?



from No symbols loaded for c++ in mixed debugging (from Python)

Equivalent of pandas merge_asof when joining spark dataframes, with merge nearest and tolerance

I'm trying to replicate pandas's merge_asof behavior when joining spark dataframes.

Let's just say I have 2 dataframe, df1 and df2:

df1 = pd.DataFrame([{"timestamp": 0.5 * i, "a": i * 2} for i in range(66)])
df2 = pd.DataFrame([{"timestamp": 0.33 * i, "b": i} for i in range(100)])

# use merge_asof to merge df1 and df2
merge_df = pd.merge_asof(df1, df2, on='timestamp', direction='nearest', tolerance=df.timestamp.diff().mean() - 1e-6)

Result after merge on merge_df would be:

timestamp a b
0.0 0 0
0.5 2 2
1.0 4 3
1.5 6 5
2.0 8 6
... ... ..
30.5 122 92
31.0 124 94
31.5 126 95
32.0 128 97
32.5 130 98

Now given similar dataframes in spark:

df1_spark = spark.createDataFrame([{"timestamp": 0.5 * i, "a": i * 2} for i in range(66)])
df2_spark = spark.createDataFrame([{"timestamp": 0.33 * i, "b": i} for i in range(100)])

How to join 2 spark dataframes to produce similar result as in pandas, with configurable direction and tolerance? Thanks.

[Edit]
As suggestion from similar posts, applying function over Window would create similar behavior to direction parameter. However, I still don't know how to apply function to find nearest row (like how nearest would behave) and within a certain range (tolerance).



from Equivalent of pandas merge_asof when joining spark dataframes, with merge nearest and tolerance

Fetch image in FirebaseMessagingService causing UnknownHostException

We are using image is our push notification. This is the code to get the bitmap from URL:

try {
        return BitmapFactory.decodeStream(URL(url).content as InputStream)
    } catch (e: Exception) {
        //Used for debugging production data only
        if (BuildConfig.FLAVOR.equals("production", true)
                && BuildConfig.BUILD_TYPE.equals("release", true)) {
            if (logData != null)
                FirebaseCrashlytics.getInstance().setCustomKey("Push Notification Test Payload", logData?.toString()
                        ?: "")
            FirebaseCrashlytics.getInstance().recordException(e)
        }
    }

Earlier it was being used directly but since it's a long running task so I moved it inside CoroutineScope to avoid NetworkOnMainThreadException as below:

CoroutineScope(Dispatchers.IO).launch {
            val bitmap = getBitmap(imagePath!!)
            val largeIcon = BitmapFactory.decodeResource(this@PushMessagingService.resources,
                    R.drawable.notification_big_icon)
            withContext(Dispatchers.Main) {
                if (bitmap != null) {
                    val notificationStyle = NotificationCompat.BigPictureStyle()
                    notificationStyle.bigPicture(bitmap)
                    notificationStyle.setSummaryText(msg)
                    buildNotification(channelId, msgTitle, msg, defaultSoundUri, contentIntent,
                            notificationStyle, largeIcon, notificationManager, notificationID)
                } else {
                    val notificationStyle = NotificationCompat.BigTextStyle()
                    buildNotification(channelId, msgTitle, msg, defaultSoundUri, contentIntent,
                            notificationStyle, largeIcon, notificationManager, notificationID)
                }
            }
        }

But now I am getting lots of UnknownHostException stack trace in Crashlytics. This is stack trace:

Non-fatal Exception: java.net.UnknownHostException: Unable to resolve host "cdn.test.com": No address associated with hostname
   at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:156)
   at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
   at java.net.InetAddress.getAllByName(InetAddress.java:1152)
   at com.android.okhttp.Dns$1.lookup(Dns.java:41)
   at com.android.okhttp.internal.http.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:178)
   at com.android.okhttp.internal.http.RouteSelector.nextProxy(RouteSelector.java:144)
   at com.android.okhttp.internal.http.RouteSelector.next(RouteSelector.java:86)
   at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:176)
   at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128)
   at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97)
   at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289)
   at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:542)
   at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:106)
   at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:30)
   at com.google.firebase.perf.network.InstrURLConnectionBase.getContent(InstrURLConnectionBase.java:86)
   at com.google.firebase.perf.network.InstrHttpsURLConnection.getContent(InstrHttpsURLConnection.java:68)
   at com.google.firebase.perf.network.FirebasePerfUrlConnection.getContent(FirebasePerfUrlConnection.java:124)
   at com.google.firebase.perf.network.FirebasePerfUrlConnection.getContent(FirebasePerfUrlConnection.java:92)
   at app.push.PushMessagingService.getBitmap(PushMessagingService.kt:254)
   at app.push.PushMessagingService.access$getBitmap(PushMessagingService.kt:35)
   at app.push.PushMessagingService$sendNotification$1.invokeSuspend(PushMessagingService.kt:145)
   at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
   at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
   at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
   at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
   at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
   
   Caused by android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
   at libcore.io.Linux.android_getaddrinfo(Linux.java)
   at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:73)
   at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:202)
   at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:73)
   at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
   at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
   at java.net.InetAddress.getAllByName(InetAddress.java:1152)
   at com.android.okhttp.Dns$1.lookup(Dns.java:41)
   at com.android.okhttp.internal.http.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:178)
   at com.android.okhttp.internal.http.RouteSelector.nextProxy(RouteSelector.java:144)
   at com.android.okhttp.internal.http.RouteSelector.next(RouteSelector.java:86)
   at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:176)
   at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128)
   at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97)
   at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289)
   at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411)
   at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:542)
   at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:106)
   at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:30)
   at com.google.firebase.perf.network.InstrURLConnectionBase.getContent(InstrURLConnectionBase.java:86)
   at com.google.firebase.perf.network.InstrHttpsURLConnection.getContent(InstrHttpsURLConnection.java:68)
   at com.google.firebase.perf.network.FirebasePerfUrlConnection.getContent(FirebasePerfUrlConnection.java:124)
   at com.google.firebase.perf.network.FirebasePerfUrlConnection.getContent(FirebasePerfUrlConnection.java:92)
   at app.push.push.PushMessagingService.getBitmap(PushMessagingService.kt:254)
   at app.push.push.PushMessagingService.access$getBitmap(PushMessagingService.kt:35)
   at app.push.push.PushMessagingService$sendNotification$1.invokeSuspend(PushMessagingService.kt:145)
   at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
   at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
   at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
   at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
   at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
   at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)

I got the server checked and its working fine. That raised me some concerns:

  1. Is anything wrong in my implementation with respect to push notifications? Since, I am not facing any issue anywhere in rest of app.
  2. What would be the efficient way to fetch bitmap from url in case of push notifications?


from Fetch image in FirebaseMessagingService causing UnknownHostException

cannot be used as a JSX component

I am working on integrating a package library named A with another package(website) named B , but in A I was getting error where it says cannot be used as a JSX component

I was be able to solve this issue by updating A package json with:

"resolutions": { "@types/react": "17.0.14", "@types/react-dom": "17.0.14" } and also added preinstall in scripts: "preinstall": "npm install --package-lock-only --ignore-scripts && npx npm-force-resolutions"

This resolved the issue, but every time I change A package which is a dependency of B, I have to remove node_modules and package-lock to have a successful build and see the new changes in B which I think it is weird anyone has any idea? here are the package.json s A, package.json

here are the package.json s

A package.json

{
  "version": "1.0.0",
  "scripts": {
    "build-storybook": "NODE_ENV=production build-storybook -c cfg/storybook -o dist/storybook",
    "build:webpack": "NODE_ENV=production webpack",
    "build": "node scripts/build.js",
    "lint:fix": "eslint -c .eslintrc.js --fix --ext js,jsx,ts,tsx src",
    "lint": "eslint -c .eslintrc.js --ext js,jsx,ts,tsx src",
    "posttest": "generate-coverage-data",
    "prepublish": "npm run typescript",
    "preinstall": "([ ! -f package-lock.json ] && npm install --package-lock-only --ignore-scripts --no-audit); npx force-resolutions",
    "release": "rm -rf dist && tsc --p tsconfig-cjs.json && npm run build",
    "scaffold": "./scripts/scaffold.sh",
    "storybook:build": "build-storybook -c ./storybook-config -o
    "storybook": "start-storybook -p 6006",
    "test:debug": "node --inspect-brk node_modules/.bin/jest --runInBand",
    "test:watch": "npm run test -- --watch",
    "test": "jest --coverage",
    "typescript": "tsc --project tsconfig.json",
    "typescript-release": "tsc --p tsconfig-cjs.json"
  },
  "keywords": [
    "react"
  ],
  "dependencies": {
    "@hookform/resolvers": "^1.3.7",
    "@react-icons/all-files": "^4.1.0",
    "@testing-library/dom": "^8.13.0",
    "@testing-library/react-hooks": "^8.0.1",
    "@wojtekmaj/enzyme-adapter-react-17": "^0.6.7",
    "emotion": "^10.0.27",
    "install": "^0.13.0",
    "is-hotkey": "^0.1.8",
    "is-url": "^1.2.4",
    "jest-environment-jsdom-sixteen": "^2.0.0",
    "lodash.clonedeep": "^4.5.0",
    "lodash.differenceby": "^4.8.0",
    "lodash.escaperegexp": "^4.1.2",
    "lodash.get": "^4.4.2",
    "lodash.isempty": "^4.4.0",
    "lodash.isequal": "^4.5.0",
    "lodash.set": "^4.3.2",
    "lodash.sortby": "^4.7.0",
    "lodash.uniqueid": "^4.0.1",
    "npm": "^8.19.2",
    "office-ui-fabric-react": "^7.107.0",
    "react-hook-form": "^6.15.7",
    "react-redux": "^8.0.2",
    "react-router-dom": "^5.2.0",
    "slate": "^0.60.8",
    "slate-history": "^0.59.0",
    "slate-react": "^0.60.8"
  },
  "resolutions": {
    "@types/react": "17.0.14",
    "@types/react-dom": "17.0.14"
  },
  "peerDependencies": {
    "react": "^16.8.6",
    "react-dom": "^16.8.6",
    "date-fns": "^2.16.1",
    "react-i18next": "^11.8.7",
    "i18next": "^19.8.9"
  },
  "devDependencies": {
    "@babel/core": "^7.6.0",
    "@babel/helper-validator-identifier": "7.10.4",
    "@babel/plugin-proposal-class-properties": "^7.12.1",
    "@babel/plugin-proposal-nullish-coalescing-operator": "^7.12.1",
    "@babel/plugin-proposal-object-rest-spread": "^7.12.1",
    "@babel/plugin-proposal-optional-chaining": "^7.12.1",
    "@babel/plugin-syntax-dynamic-import": "^7.8.3",
    "@babel/plugin-transform-react-jsx": "^7.12.1",
    "@babel/plugin-transform-runtime": "^7.12.1",
    "@babel/plugin-transform-typescript": "^7.12.1",
    "@babel/preset-env": "^7.12.1",
    "@babel/preset-react": "^7.12.1",
    "@babel/preset-typescript": "^7.12.1",
    "@babel/runtime": "7.4.5",
    "@sheerun/mutationobserver-shim": "^0.3.3",
    "@storybook/addon-actions": "^6.0.26",
    "@storybook/addon-essentials": "^6.0.26",
    "@storybook/addon-info": "5.3.21",
    "@storybook/addon-knobs": "^6.0.26",
    "@storybook/addon-links": "^6.0.26",
    "@storybook/addons": "^6.0.26",
    "@storybook/react": "^6.0.26",
    "@testing-library/jest-dom": "^5.16.1",
    "@testing-library/react": "^12.1.2",
    "@types/date-fns": "^2.6.0",
    "@types/enzyme": "^3.10.7",
    "@types/history": "^4.7.8",
    "@types/is-url": "^1.2.4",
    "@types/jest": "^26.0.24",
    "@types/lodash.clonedeep": "^4.5.6",
    "@types/lodash.differenceby": "^4.8.6",
    "@types/lodash.get": "^4.4.6",
    "@types/lodash.isempty": "^4.4.6",
    "@types/lodash.isequal": "^4.5.5",
    "@types/lodash.set": "^4.3.6",
    "@types/lodash.sortby": "^4.7.6",
    "@types/react": "^17.0.14",
    "@types/react-dom": "^17.0.14",
    "@types/react-router-dom": "^5.2.0",
    "@types/storybook-react-router": "^1.0.1",
    "@typescript-eslint/eslint-plugin": "^4.4.0",
    "@typescript-eslint/parser": "^4.4.0",
    "babel-eslint": "^10.1.0",
    "babel-loader": "^8.1.0",
    "babel-plugin-module-resolver": "^3.1.1",
    "babel-plugin-transform-react-remove-prop-types": "^0.4.19",
    "date-fns": "^2.16.1",
    "enzyme": "^3.11.0",
    "enzyme-adapter-react-16": "^1.15.6",
    "eslint": "7.11.0",
    "eslint-config-prettier": "^7.0.0",
    "eslint-plugin-jest": "^24.1.0",
    "eslint-plugin-react": "^7.21.5",
    "file-loader": "^6.2.0",
    "i18next": "^19.8.9",
    "react-i18next": "^11.8.7",
    "fs-extra": "^5.0.0",
    "globby": "^8.0.1",
    "history": "^4.10.1",
    "husky": "^4.3.8",
    "identity-obj-proxy": "^3.0.0",
    "jest": "^24.9.0",
    "jest-environment-enzyme": "^7.1.2",
    "jest-enzyme": "^7.1.2",
    "jest-transform-stub": "^2.0.0",
    "jest-when": "^3.2.1",
    "lint-staged": "^10.5.3",
    "prettier": "^2.2.1",
    "prettier-plugin-organize-imports": "^1.1.1",
    "react-to-typescript-definitions": "^1.2.0",
    "storybook-react-router": "^1.0.8",
    "typescript": "4.7.4",
    "webpack-cli": "^4.2.0"
  },
  "npm-pretty-much": {
    "publishDir": "",
    "runRelease": "always"
  },
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
}

B package.json:

{
  "version": "1.0.0",
  "license": "UNLICENSED",
  "files": [],
  "scripts": {
    "check-types": "tsc --project tsconfig.json",
    "cypress": "cypress install && cypress run --env configFile=dev",
    "cypress:open": "cypress open --env configFile=dev",
    "cypress-beta": "cypress install && cypress info && cypress run --env configFile=beta || (npm run postcypress ; exit 1)",
    "cypress-beta:open": "cypress open --env configFile=beta",
    "cypress-gamma": "cypress install && cypress run --env configFile=gamma || (npm run postcypress ; exit 1)",
    "cypress-gamma:open": "cypress open --env configFile=gamma",
    "postcypress": "npm run merge-reports",
    "posttest": "if command -v generate-coverage-data >/dev/null 2>&1; then generate-coverage-data; fi",
    "merge-reports": "node ./scripts/merge-reports.js",
    "clean": "rm -rf $(readlink build) && rm -rf build coverage",
    "createmons": "createmons",
    "lint": "eslint -c .eslintrc.js --ext js,jsx,ts,tsx src",
    "lint-fix": "eslint -c .eslintrc.js --fix --ext js,jsx,ts,tsx src",
    "lint-fix-prettier": "npm run lint-fix && npm run prettier",
    "listcloudformationoutputs": "listcloudformationoutputs",
    "build": "npm run lint-fix-prettier && NODE_ENV=production webpack --config configuration/webpack.config && node ./configuration/prepare-cdk-files",
    "server": "SERVER_MODE=standalone webpack serve --config configuration/webpack.config",
    "mons-server": "webpack serve --config configuration/webpack.config",
    "secure-mons-server": "webpack serve --config configuration/webpack.config --https --key ~/cert/keyFile.key --cert ~/cert/certFile.crt",
    "prepublishOnly": "npm run build",
    "prettier": "./node_modules/.bin/prettier --write 'src/**/*@(.js|.ts|.tsx)'",
    "test": "jest",
    "test-watch": "jest --watch",
    "test:debug": "node --inspect --trace-warnings node_modules/.bin/jest --runInBand",
    "test:strict": "node --trace-warnings --unhandled-rejections=strict node_modules/.bin/jest --runInBand",
    "browser-test": "katal-integration-test --nightwatchBaseConfig browser-tests/nightwatch/nightwatch.base.json",
    "browser-test-regenerate-screenshots": "REGENERATE_SCREENSHOTS=1 npm run browser-test -- --url http://$(hostname):4321",
    "browser-test-local": "npm run browser-test -- --url http://$(hostname):4321",
    "browser-test-beta-na": "npm run browser-test -- --url https://rainier-m1k.integ.amazon.com/colombiasampleapp/index.html --materialSetName katal.integration.credentials.betaEMCTestAccount --merchantId A2KAGNIRNSGDD --marketplaceId ATVPDKIKX0DER [this is an example. use your own app url]",
    "browser-test-beta-eu": "npm run browser-test -- --url [url here] --materialSetName katal.integration.credentials.betaEMCTestAccount",
    "browser-test-beta-fe": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-beta-cn": "npm run browser-test -- --url [url here]--materialSetName [material set here]",
    "browser-test-gamma-na": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-gamma-eu": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-gamma-fe": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-gamma-cn": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-prod-na": "npm run browser-test -- --url [url here]--materialSetName [material set here]",
    "browser-test-prod-eu": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-prod-fe": "npm run browser-test -- --url [url here] --materialSetName [material set here]",
    "browser-test-prod-cn": "npm run browser-test -- --url [url here] --materialSetName [material set here]"
  },
  "lint-staged": {
    "src/**/*.{js,jsx,ts,tsx}": "eslint --cache --fix",
    "src/**/*.{js,ts,jsx,tsx,css,md}": "prettier --write"
  },
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "dependencies": {
    "@apollo/client": "3.1.3",
    "@babel/core": "^7.9.0",
    "@babel/plugin-proposal-class-properties": "^7.8.3",
    "@babel/plugin-transform-runtime": "^7.8.3",
    "@babel/preset-env": "^7.9.5",
    "@babel/preset-react": "^7.9.4",
    "@babel/preset-typescript": "^7.9.0",
    "@babel/register": "^7.8.6",
    "@babel/runtime": "~7.12.18",
    "@testing-library/jest-dom": "^5.9.0",
    "@testing-library/react": "^10.0.4",
    "@testing-library/react-hooks": "^3.2.1",
    "@types/enzyme": "^3.10.8",
    "@types/enzyme-adapter-react-16": "^1.0.6",
    "@types/jest": "^26.0.20",
    "@types/lodash": "^4.14.149",
    "@types/node": "^14.14.34",
    "@types/react": "^16.8.10",
    "@types/react-dom": "^16.8.3",
    "@types/react-redux": "^7.1.7",
    "@types/react-router-dom": "^5.1.3",
    "@types/react-transition-group": "4.2.4",
    "@types/redux-logger": "^3.0.7",
    "@typescript-eslint/eslint-plugin": "^4.4.0",
    "@typescript-eslint/parser": "^4.4.0",
    "autoprefixer": "^10.2.4",
    "aws-sdk": "^2.628.0",
    "axios": "^0.21.1",
    "babel-jest": "^26.6.3",
    "babel-loader": "^8.2.2",
    "clean-webpack-plugin": "^3.0.0",
    "core-js": "^3.9.0",
    "cross-fetch": "3.1.4",
    "css-loader": "^6.0.2",
    "css-minimizer-webpack-plugin": "^3.0.0",
    "cypress": "^7.0.0",
    "date-fns": "^2.16.1",
    "emotion": "^10.0.27",
    "enzyme": "^3.11.0",
    "enzyme-adapter-react-16": "^1.15.6",
    "eslint": "7.11.0",
    "eslint-config-prettier": "^7.0.0",
    "eslint-plugin-jest": "^24.7.0",
    "eslint-plugin-react": "^7.21.5",
    "file-loader": "^6.2.0",
    "fork-ts-checker-webpack-plugin": "^6.1.0",
    "graphql": "^14.6.0",
    "graphql-tag": "^2.10.3",
    "html-webpack-plugin": "^5.2.0",
    "husky": "^2.4.0",
    "i18next": "^19.8.9",
    "i18next-browser-languagedetector": "^6.0.1",
    "i18next-icu": "^2.0.0",
    "i18next-xhr-backend": "^3.2.2",
    "identity-obj-proxy": "^3.0.0",
    "intl-messageformat": "^9.4.7",
    "jest": "^26.6.3",
    "jest-enzyme": "^7.1.2",
    "jest-transform-stub": "^2.0.0",
    "js-yaml": "^4.0.0",
    "junit-report-merger": "^1.0.0",
    "lodash": "^4.17.11",
    "mini-css-extract-plugin": "^1.3.8",
    "mocha": "^3.2.0",
    "mocha-junit-reporter": "^1.13.0",
    "mocha-multi-reporters": "^1.1.7",
    "mochawesome": "^3.1.1",
    "mochawesome-merge": "^1.0.7",
    "mochawesome-report-generator": "^3.1.5",
    "nightwatch": "^1.5.0",
    "office-ui-fabric-react": "7.107.0",
    "optimize-css-assets-webpack-plugin": "^5.0.4",
    "otpauth": "^7.0.7",
    "postcss": "^8.1.0",
    "postcss-custom-media": "8.0.0",
    "postcss-flexbugs-fixes": "^5.0.2",
    "postcss-loader": "^6.0.3",
    "postcss-preset-env": "^6.6.0",
    "prettier-plugin-organize-imports": "^1.1.1",
    "react": "^16.8.6",
    "react-dom": "^16.8.6",
    "react-error-boundary": "^3.1.0",
    "react-hot-loader": "^4.12.21",
    "react-i18next": "^11.8.7",
    "react-intl": "^5.12.3",
    "react-redux": "^7.2.0",
    "react-router-dom": "^5.1.2",
    "react-test-renderer": "^16.8.6",
    "react-transition-group": "4.3.0",
    "redux": "^4.0.5",
    "redux-logger": "^3.0.6",
    "redux-mock-store": "^1.5.4",
    "redux-thunk": "^2.3.0",
    "sass": "^1.32.8",
    "sass-loader": "^12.0.1",
    "style-loader": "^2.0.0",
    "tachyons": "^7.0.1-5",
    "terser-webpack-plugin": "^5.1.1",
    "typescript": "4.7.4",
    "url-loader": "^4.1.1",
    "webpack": "^5.23.0",
    "webpack-cli": "^4.5.0",
    "webpack-dev-server": "^3.11.2"
  },
  "devDependencies": {
    "copy-webpack-plugin": "^5.1.1",
    "lint-staged": "^10.5.3",
    "prettier": "^2.2.1",
    "redux-devtools-extension": "^2.13.8"
  }
}

will appreciate your help



from cannot be used as a JSX component

How to find a distribution function from the max, min and average of a sample

Given that I know the, Max, Min and Average of sample (I don't have access to the sample itself). I would like to write a generic function to generate a sample with the same characteristics. From this answer I gather that this is no simple task since many distribuitions can be found with the same characteristics.

max, min, average = [411, 1, 20.98]

I'm trying to use scipy.norm but unsuccessfully. I can't seem to understand if I can pass the arguments mentioned above or if they are just returned values from an already generated function. I'm pretty new to python stats so this might be something quite easy to solve.



from How to find a distribution function from the max, min and average of a sample

Add columns to a table or records without duplicates in Duckdb

I have the following code:

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler, PatternMatchingEventHandler
import duckdb

path = "landing/persistent/"
global con
con = duckdb.connect(database='formatted/my-db.duckdb', read_only=False)

class EventHandler(PatternMatchingEventHandler):
    def __init__(self, query):
        PatternMatchingEventHandler.__init__(
            self,
            patterns=["*.csv"],
            ignore_directories=True,
            case_sensitive=False,
        )
        self.query = query
    
    
    def on_created(self, event):
        filename = event.src_path.split("/")[-1]
        filename_trunc = filename.split(".")[0]
        try:
            cursor = con.cursor()
            cursor.execute(query.format(filename_trunc), 
                           [event.src_path])
            
            cursor.execute("show tables")
            print(cursor.fetchall())
        except Exception as e:
            print(e)
        finally:
            cursor.close()

query = "CREATE TABLE {} AS SELECT * FROM read_csv_auto(?);"
event_handler = EventHandler(query)
observer = Observer()
observer.schedule(event_handler, path, recursive=True)
observer.start()

try:
    while True:
        time.sleep(1)
        
except KeyboardInterrupt:
    observer.stop()
observer.join()

It basically waits for a file in the path and adds it to the db as a new table. However, I have two extra use cases:

  • Add a new csv with the some modifications (with a table already created for that filename) and maybe some rows already exist in the table.
  • Add a new csv with a new column (with a table already created for that filename)

However, I don't know how efficiently keep track of these two cases using sql nor duckdb. Any help would be appreciated, thanks.



from Add columns to a table or records without duplicates in Duckdb

Wednesday 26 October 2022

Apple MusicKit SDK in Android

I have been trying to add Apple MusicKit auth SDK to an android app, I have added the AAR provided by apple and the necessary step to initialize the auth flow, but the SDK just redirects me to the google play store listing of apple music, I do have the app installed and logged in on my test device.

Apple MusicKit Auth SDK : musickitauth-release-1.1.2.aar

Here's the piece of code I use to init the auth flow :

val intent = authenticationManager.createIntentBuilder(BuildConfig.APPLE_DEV_TOKEN)
    .setHideStartScreen(true)
    .setStartScreenMessage("Connect with Apple Music")
    .build()

startActivityForResult(intent , Const.APPLE_REQ_CODE)

Here's the ActivityRequest Code :

val result = authenticationManager.handleTokenResult(data)

if (result.isError) {
    val error = result.error
    Alerts.log(TAG , "APPLE ERROR: $error")
}
else {
    Alerts.log(TAG , "APPLE TOKEN: ${result.musicUserToken}")
}

This is what I get in my logs :

D/SDKUriHandlerActivity: onCreate: DEV_TOKEN_HERE
D/AuthUtils: deeplinkAppleMusic: uri = musicsdk://applemusic/authenticate-v1?appPackage=PACKAGE_ID&devToken=DEV_TOKEN_HERE
W/System.err: android.content.pm.PackageManager$NameNotFoundException: com.apple.android.music
W/System.err:     at android.app.ApplicationPackageManager.getPackageInfoAsUser(ApplicationPackageManager.java:244)
W/System.err:     at android.app.ApplicationPackageManager.getPackageInfo(ApplicationPackageManager.java:213)
W/System.err:     at com.apple.android.sdk.authentication.a.a(AuthSDK:92)
W/System.err:     at com.apple.android.sdk.authentication.a.b(AuthSDK:66)
W/System.err:     at com.apple.android.sdk.authentication.a.b(AuthSDK:56)
W/System.err:     at com.apple.android.sdk.authentication.SDKUriHandlerActivity.onCreate(AuthSDK:43)
W/System.err:     at android.app.Activity.performCreate(Activity.java:8183)
W/System.err:     at android.app.Activity.performCreate(Activity.java:8167)
W/System.err:     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1316)
W/System.err:     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3751)
W/System.err:     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3950)
W/System.err:     at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
W/System.err:     at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
W/System.err:     at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
W/System.err:     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2377)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:106)
W/System.err:     at android.os.Looper.loop(Looper.java:262)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:8304)
W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
W/System.err:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:632)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1049)
D/a: deeplinkAppleMusic: isAppleMusicInstalled(activity) = 2
W/System.err: android.content.pm.PackageManager$NameNotFoundException: com.apple.android.music
W/System.err:     at android.app.ApplicationPackageManager.getPackageInfoAsUser(ApplicationPackageManager.java:244)
W/System.err:     at android.app.ApplicationPackageManager.getPackageInfo(ApplicationPackageManager.java:213)
W/System.err:     at com.apple.android.sdk.authentication.a.a(AuthSDK:92)
W/System.err:     at com.apple.android.sdk.authentication.a.b(AuthSDK:67)
W/System.err:     at com.apple.android.sdk.authentication.a.b(AuthSDK:56)
W/System.err:     at com.apple.android.sdk.authentication.SDKUriHandlerActivity.onCreate(AuthSDK:43)
W/System.err:     at android.app.Activity.performCreate(Activity.java:8183)
W/System.err:     at android.app.Activity.performCreate(Activity.java:8167)
W/System.err:     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1316)
W/System.err:     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3751)
W/System.err:     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3950)
W/System.err:     at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
W/System.err:     at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
W/System.err:     at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
W/System.err:     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2377)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:106)
W/System.err:     at android.os.Looper.loop(Looper.java:262)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:8304)
W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
W/System.err:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:632)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1049)
D/AuthUtils: deeplinkAppleMusic: activityNotFound!
D/AuthUtils: sendMarketIntent : uri = musicsdk://applemusic/authenticate-v1?appPackage=PACKAGE_ID&devToken=DEV_TOKEN_HERE
D/AuthUtils: sendMarketIntent: uri = market://details?id=com.apple.android.music&referrer=musicsdk%3A%2F%2Fapplemusic%2Fauthenticate-v1%3FappPackage%3DPACKAGE_ID%26devToken%DEV_TOKEN_HERE
V/PhoneWindow: DecorView setVisiblity: visibility = 4, Parent = null, this = DecorView@dd295ee[]
D/ViewRootImpl[SDKUriHandlerActivity]: hardware acceleration = true , fakeHwAccelerated = false, sRendererDisabled = false, forceHwAccelerated = false, sSystemRendererDisabled = false

There isn't much documentation for this anywhere, the apple site just refers to the code references and the AAR download site.

SDK Manifest :

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.apple.android.sdk.authentication"
    android:versionCode="1"
    android:versionName="1.2.0" >

    <uses-sdk
        android:minSdkVersion="21"
        android:targetSdkVersion="28" />

    <application
        android:allowBackup="true"
        android:supportsRtl="true" >
        <activity
            android:name="com.apple.android.sdk.authentication.StartAuthenticationActivity"
            android:screenOrientation="portrait"
            android:theme="@style/MusicKitAuthTheme" >
        </activity>
        <activity
            android:name="com.apple.android.sdk.authentication.SDKUriHandlerActivity"
            android:launchMode="singleTask"
            android:screenOrientation="portrait"
            android:exported="true"
            android:theme="@style/MusicKitAuthTheme" >
            <intent-filter>
                <action android:name="android.intent.action.VIEW" />

                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />

                <data
                    android:host="${applicationId}"
                    android:pathPattern="/authenticateresult.*"
                    android:scheme="musicsdk" />
            </intent-filter>
        </activity>
    </application>

    <queries>
        <intent>
            <action android:name="android.intent.action.VIEW" />
            <data android:scheme="com.apple.android.music" />
        </intent>
    </queries>

</manifest>


from Apple MusicKit SDK in Android

If I click on prev,next,today calendar, I want to get prev data calendar, how is that?

image clik -> If I click on prev,next,today I want to get data with calender.I am getting many errors.How to fix this error?

prev.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            googleMap.clear();
            Date date1=new Date(System.currentTimeMillis() - (1000 * 60 * 60 * 24) * minusButtonPressed);
            SimpleDateFormat sdf=new SimpleDateFormat("dd-MMM-yyyy");
            String date=sdf.format(date1);
            drawRoute(date);
            calculateDistance(date);
            minusButtonPressed += 1;
        }
    });

    next.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            googleMap.clear();
            Date date1=new Date(System.currentTimeMillis() + (1000 * 60 * 60 * 24) * plusButtonPressed);
            SimpleDateFormat sdf=new SimpleDateFormat("dd-MMM-yyyy");
            String date=sdf.format(date1);
            drawRoute(date);
            calculateDistance(date);
            plusButtonPressed += 1;
        }
    });

rest api click path How to set restapi id if calendar prev next is clicked callbackCall = apiInterface.getRecipesByCategory(date, AppConfig.REST_API_KEY);

private void requestHomeData2() {
    this.callbackCall = RestAdapter.createAPI(sharedPref.getApiUrl()).getHome(AppConfig.REST_API_KEY);
    callbackCall = apiInterface.getRecipesByCategory(date, AppConfig.REST_API_KEY);

    this.callbackCall.enqueue(new Callback<CallbackHome>() {
        public void onResponse(Call<CallbackHome> call, Response<CallbackHome> response) {
            CallbackHome responseHome = response.body();
            if (responseHome == null || !responseHome.status.equals("ok")) {
                onFailRequest();
                return;
            }
            displayData(responseHome);
            swipeProgress(false);
            lyt_main_content.setVisibility(View.VISIBLE);
        }

        public void onFailure(Call<CallbackHome> call, Throwable th) {
            Log.e("onFailure", th.getMessage());
            if (!call.isCanceled()) {
                onFailRequest();
            }
        }
    });


from If I click on prev,next,today calendar, I want to get prev data calendar, how is that?

Show scrollbars on iOS when scrolling via Javascript

I would like the vertical scrollbar to be displayed when I call element.scrollTop = <somePosition>;

Here is an example of what I'm trying to achieve: https://plnkr.co/edit/0ls05cCa3XrvrWPO?open=lib%2Fscript.js&preview

If that example is loaded on iOS, when the left box is scrolled, the right box should sync with it. That works just fine, but I need a way for the scrollbar to also show up.



from Show scrollbars on iOS when scrolling via Javascript

How do you calculate spherical camera position from point correspondence?

I have 4 points marked in an equirectangular image. [Red dots]

enter image description here

I also have the 4 corresponding points marked in an overhead image [ Red dots ]

enter image description here

How do I calculate where on the overhead image the camera was positioned?

So far I see there are 4 rays (R1, R2, R3, R4) extending from the unknown camera center C = (Cx, Cy, Cz) through the points in the equirectangular image and ending at the pixel coordinates of the overhead image (P1, P2, P3, P4). So 4 vector equations of the form:

[Cx, Cy, Cz] + [Rx, Ry, Rz]*t = [x, y, 0] 

for each correspondence. So

C + R1*t1 = P1 = [x1, y1, 0]
C + R2*t2 = P2 = [x2, y2, 0]
C + R3*t3 = P3 = [x3, y3, 0]
C + R4*t4 = P4 = [x4, y4, 0]

So 7 unknowns and 12 equations? This was my attempt but doesn't seem to give a reasonable answer:

import numpy as np

def equi2sphere(x, y):
    width = 2000
    height = 1000
    theta = 2 * np.pi * x  / width - np.pi
    phi = np.pi * y / height
    return theta, phi

HEIGHT = 1000
MAP_HEIGHT = 788
#
# HEIGHT = 0
# MAP_HEIGHT = 0

# Point in equirectangular image, bottom left = (0, 0)
xs = [1190, 1325, 1178, 1333]
ys = [HEIGHT - 730,   HEIGHT - 730,  HEIGHT - 756,  HEIGHT - 760]

# import cv2
# img = cv2.imread('equirectangular.jpg')
# for x, y in zip(xs, ys):
#     img = cv2.circle(img, (x, y), 15, (255, 0, 0), -1)
# cv2.imwrite("debug_equirectangular.png", img)

# Corresponding points in overhead map, bottom left = (0, 0)
px = [269, 382, 269, 383]
py = [778, 778, 736, 737]

# import cv2
# img = cv2.imread('map.png')
# for x, y in zip(px, py):
#     img = cv2.circle(img, (x, y), 15, (255, 0, 0), -1)
# cv2.imwrite("debug_map.png", img)

As = []
bs = []
for i in range(4):

    x, y = xs[i], ys[i]

    theta, phi = equi2sphere(x, y)

    # convert to spherical
    p = 1
    sx = p * np.sin(phi) * np.cos(theta)
    sy = p * np.sin(phi) * np.sin(theta)
    sz = p * np.cos(phi)

    print(x, y, '->', np.degrees(theta), np.degrees(phi), '->', round(sx, 2), round(sy, 2), round(sz, 2))

    block = np.array([
        [1, 0, 0, sx],
        [0, 1, 0, sy],
        [1, 0, 1, sz],
    ])

    y = np.array([px[i], py[i], 0])

    As.append(block)
    bs.append(y)



A = np.vstack(As)
b = np.hstack(bs).T
solution = np.linalg.lstsq(A, b)
Cx, Cy, Cz, t = solution[0]

import cv2
img = cv2.imread('map_overhead.png')

for i in range(4):

    x, y = xs[i], ys[i]

    theta, phi = equi2sphere(x, y)

    # convert to spherical
    p = 1
    sx = p * np.sin(phi) * np.cos(theta)
    sy = p * np.sin(phi) * np.sin(theta)
    sz = p * np.cos(phi)

    pixel_x = Cx + sx * t
    pixel_y = Cy + sy * t
    pixel_z = Cz + sz * t
    print(pixel_x, pixel_y, pixel_z)
    img = cv2.circle(img, (int(pixel_x), img.shape[0] - int(pixel_y)), 15, (255,255, 0), -1)
    img = cv2.circle(img, (int(Cx), img.shape[0] - int(Cy)), 15, (0,255, 0), -1)


cv2.imwrite("solution.png", img)


# print(A.dot(solution[0]))
# print(b)

Resulting camera position (Green) and projected points (Teal)

enter image description here

EDIT: One bug fixed is that the longitude offset in the equirectangular images in PI/4 which fixes the rotation issue but the scale is still off somehow.



from How do you calculate spherical camera position from point correspondence?