Tuesday 31 August 2021

How to position the cursor at the beginning of the mask when pushing tab key?

In the form in the phone field, I use a mask based on the Imask library.

  1. I am trying to make it when focusing with both the mouse or the tab on the field, a full mask with brackets appears, and the cursor is set to the initial input position. There is a problem here: either an incomplete mask appears, or only +38, or the cursor moves to the end due to a taboo on the field.
  2. I can't clear the phone field after submitting the form, I get warning:

Element value was changed outside of mask. Syncronize mask using mask.updateValue() to work properly.

const phone = document.querySelector('.phone');
const name = document.querySelector('.name');
const form = document.querySelector('#form');

if (phone) {
  phone.onfocus = () => {
    newIMask(phone)
  }

  form.addEventListener('keydown', function(e) {
    if (e.key == 'Tab') {
      if (phone.value === '') {
        let tabMask = IMask(
          phone, {
            mask: '+38( 000) 000-00-00',
            lazy: false
          });
      }
    }
  });
}

function newIMask(phone) {
  let phoneMask = IMask(
    phone, {
      mask: '{+38} (000) 000-00-00',
      lazy: false
    });
  phone.value = phoneMask.unmaskedValue; //если закомментировать, то по табу курсор сместиться вконец
}

document.getElementById("form").addEventListener('submit', function(e) {
  e.preventDefault();
  phone.value = "";
  window.open('mailto:mail@example.com?name=' + name.value + '&body=' + phone.value);
});
<script src="https://unpkg.com/imask"></script>
<form id="form">
  <input type="text" name="name" class="name">
  <input type="tel" name="phone" class="phone">
  <input type="submit" value="Отправить">
  <form>

UPD With this insertion of the code, I sort of solve the issue with the tab

const form = document.querySelector('#form');
    form.addEventListener('keydown', function (e) {
      if(e.key == 'Tab') {
        if(phone.value === '') {
          let tabMask = IMask(
            phone, {
              mask: '+38( 000) 000-00-00',
              lazy: false
          });
        }
      }
    });

but each digit is entered with a warring:

Warring: Element value was changed outside of mask. Syncronize mask using mask.updateValue() to work properly.

and the phone field is not cleared when sending.

Could you help me with finding the right solution?



from How to position the cursor at the beginning of the mask when pushing tab key?

VS Code integrated terminal is messing up the clipboard python code when pasting

In a clean installation of VS Code (1.59.1) the code pasted in the integrated terminal has a different indentation than the one available in the editor. I tried changing the parameter "editor.formatOnPaste": true and there was no improvement. This problem does not exist when I copy the same content and paste it into the powershell or command prompt. Any ideas on how to solve this?

error description

enter image description here



from VS Code integrated terminal is messing up the clipboard python code when pasting

D3.js highlight related nodes / links

Question: I want to fade / highlight a whole dependency chain, based on the link type.

To do so I utilize the mouseEnter event, which currently store all links and nodes. Further I fade all nodes and links and only highlight those nodes which where filtered as related nodes and links. It would require to check all related nodes and links again, if those have connections from type need too. This must be done as long as dependency connections are found.. I can´t figure out a proper algorythm.

Examples:

For better understanding I created a beer ingredients dependency, which looks lika a star. For those purposes my version is fine. BUT the second chain, about car -> wheel -> tires -> rubber and the radio is giving me headache. The radio is a "use" dependency, means its not mandatory for the chain and shouldn´t be hightlighted.

Expected result:

If the cursor is over car all connected nodes with a "need" dependency should be highlighted and the rest should fade.

For those who wants to help me with, please dont hesitate to ask, if anything is unclear.

<!DOCTYPE html>
<html>

<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <!-- d3.js framework -->
    <script src="https://d3js.org/d3.v6.js"></script>
    <!-- fontawesome stylesheet https://fontawesome.com/ -->
    <script src="https://kit.fontawesome.com/39094309d6.js" crossorigin="anonymous"></script>
</head>

<style>
    body {
        height: 100%;
        background: #e6e7ee;
        overflow: hidden;
        margin: 0px;
    }

    .faded {
        opacity: 0.1;
        transition: 0.3s opacity;
    }

    .highlight {
        opacity: 1;
    }
</style>

<body>
    <svg id="svg"></svg>

    <script>
        var graph = {
            "nodes": [
                {
                    "id": 0,
                    "name": "beer",
                },
                {
                    "id": 1,
                    "name": "water",
                },
                {
                    "id": 2,
                    "name": "hop",
                },
                {
                    "id": 3,
                    "name": "malt",
                },
                {
                    "id": 4,
                    "name": "yeast",
                },
                {
                    "id": 10,
                    "name": "car",
                },
                {
                    "id": 11,
                    "name": "wheels",
                },
                {
                    "id": 12,
                    "name": "tires",
                },
                {
                    "id": 13,
                    "name": "rubber",
                },
                {
                    "id": 14,
                    "name": "radio",
                }
            ],
            "links": [
                {
                    "source": 0,
                    "target": 1,
                    "type": "need"
                },
                {
                    "source": 0,
                    "target": 2,
                    "type": "need"
                },
                {
                    "source": 0,
                    "target": 3,
                    "type": "need"
                },
                {
                    "source": 0,
                    "target": 4,
                    "type": "need"
                },
                {
                    "source": 10,
                    "target": 11,
                    "type": "need"
                },
                {
                    "source": 11,
                    "target": 12,
                    "type": "need"
                },
                {
                    "source": 12,
                    "target": 13,
                    "type": "need"
                },
                {
                    "source": 10,
                    "target": 14,
                    "type": "use"
                }

            ]
        }

        var svg = d3.select("svg")
            .attr("class", "canvas")
            .attr("width", window.innerWidth)
            .attr("height", window.innerHeight)
            .call(d3.zoom().on("zoom", function (event) {
                svg.attr("transform", event.transform)
            }))
            .append("g")

        // append markers to svg
        svg.append("defs").append("marker")
            .attr("id", "arrowhead")
            .attr("viewBox", "-0 -5 10 10")
            .attr("refX", 8)
            .attr("refY", 0)
            .attr("orient", "auto")
            .attr("markerWidth", 50)
            .attr("markerHeight", 50)
            .attr("xoverflow", "visible")
            .append("svg:path")
            .attr("d", "M 0,-1 L 2 ,0 L 0,1")
            .attr("fill", "black")
            .style("stroke", "none")

        var linksContainer = svg.append("g").attr("class", linksContainer)
        var nodesContainer = svg.append("g").attr("class", nodesContainer)

        var force = d3.forceSimulation()
            .force("link", d3.forceLink().id(function (d) {
                return d.id
            }).distance(80))
            .force("charge", d3.forceManyBody().strength(-100))
            .force("center", d3.forceCenter(window.innerWidth / 2, window.innerHeight / 2))
            .force("collision", d3.forceCollide().radius(90))

        initialize()

        function initialize() {

            link = linksContainer.selectAll(".link")
                .data(graph.links)
                .join("line")
                .attr("class", "link")
                .attr('marker-end', 'url(#arrowhead)')
                .style("display", "block")
                .style("stroke", "black")
                .style("stroke-width", 1)

            linkPaths = linksContainer.selectAll(".linkPath")
                .data(graph.links)
                .join("path")
                .style("pointer-events", "none")
                .attr("class", "linkPath")
                .attr("fill-opacity", 1)
                .attr("stroke-opacity", 1)
                .attr("id", function (d, i) { return "linkPath" + i })
                .style("display", "block")

            linkLabels = linksContainer.selectAll(".linkLabel")
                .data(graph.links)
                .join("text")
                .style("pointer-events", "none")
                .attr("class", "linkLabel")
                .attr("id", function (d, i) { return "linkLabel" + i })
                .attr("font-size", 16)
                .attr("fill", "black")
                .text("")

            linkLabels
                .append("textPath")
                .attr('xlink:href', function (d, i) { return '#linkPath' + i })
                .style("text-anchor", "middle")
                .style("pointer-events", "none")
                .attr("startOffset", "50%")
                .text(function (d) { return d.type })

            node = nodesContainer.selectAll(".node")
                .data(graph.nodes, d => d.id)
                .join("g")
                .attr("class", "node")
                .call(d3.drag()
                    .on("start", dragStarted)
                    .on("drag", dragged)
                    .on("end", dragEnded)
                )

            node.selectAll("circle")
                .data(d => [d])
                .join("circle")
                .attr("r", 30)
                .style("fill", "whitesmoke")
                .on("mouseenter", mouseEnter)
                .on("mouseleave", mouseLeave)

            node.selectAll("text")
                .data(d => [d])
                .join("text")
                .style("class", "icon")
                .attr("font-family", "FontAwesome")
                .attr("dominant-baseline", "central")
                .attr("text-anchor", "middle")
                .attr("font-size", 20)
                .attr("fill", "black")
                .attr("pointer-events", "none")
                .attr("dy", "-1em")
                .text(function (d) {
                    return d.name
                })
            node.append("text")
                .attr("dominant-baseline", "central")
                .attr("text-anchor", "middle")
                .attr("font-size", 13)
                .attr("fill", "black")
                .attr("pointer-events", "none")
                .attr("dy", "0.5em")
                .text(function (d) {
                    return d.id
                })

            force
                .nodes(graph.nodes)
                .on("tick", ticked);

            force
                .force("link")
                .links(graph.links)
        }

        function mouseEnter(event, d) {
            const selNodes = node.selectAll("circle")
            const selLink = link
            const selLinkLabel = linkLabels
            const selText = node.selectAll("text")
            const related = []
            const relatedLinks = []

            related.push(d)
            force.force('link').links().forEach((link) => {
                if (link.source === d || link.target === d) {
                    relatedLinks.push(link)
                    if (related.indexOf(link.source) === -1) { related.push(link.source) }
                    if (related.indexOf(link.target) === -1) { related.push(link.target) }
                }
            })
            selNodes.classed('faded', true)
            selNodes.filter((dNodes) => related.indexOf(dNodes) > -1)
                .classed('highlight', true)
            selLink.classed('faded', true)
            selLink.filter((dLink) => dLink.source === d || dLink.target === d)
                .classed('highlight', true)
            selLinkLabel.classed('faded', true)
            selLinkLabel.filter((dLinkLabel) => dLinkLabel.source === d || dLinkLabel.target === d)
                .classed('highlight', true)
            selText.classed('faded', true)
            selText.filter((dText) => related.indexOf(dText) > -1)
                .classed('highlight', true)
            
            force.alphaTarget(0.0001).restart()
        }

        function mouseLeave(event, d) {
            const selNodes = node.selectAll("circle")
            const selLink = link
            const selLinkLabel = linkLabels
            const selText = node.selectAll("text")

            selNodes.classed('faded', false)
            selNodes.classed('highlight', false)
            selLink.classed('faded', false)
            selLink.classed('highlight', false)
            selLinkLabel.classed('faded', false)
            selLinkLabel.classed('highlight', false)
            selText.classed('faded', false)
            selText.classed('highlight', false)
            
            force.restart()
        }

        function ticked() {
            // update link positions
            link
                .attr("x1", function (d) {
                    return d.source.x;
                })
                .attr("y1", function (d) {
                    return d.source.y;
                })
                .attr("x2", function (d) {
                    return d.target.x;
                })
                .attr("y2", function (d) {
                    return d.target.y;
                });

            // update node positions
            node
                .attr("transform", function (d) {
                    return "translate(" + d.x + ", " + d.y + ")";
                });

            linkPaths.attr('d', function (d) {
                return 'M ' + d.source.x + ' ' + d.source.y + ' L ' + d.target.x + ' ' + d.target.y;
            });

            linkLabels.attr('transform', function (d) {
                if (d.target.x < d.source.x) {
                    var bbox = this.getBBox();

                    rx = bbox.x + bbox.width / 2;
                    ry = bbox.y + bbox.height / 2;
                    return 'rotate(180 ' + rx + ' ' + ry + ')';
                }
                else {
                    return 'rotate(0)';
                }
            });

        }

        function dragStarted(event, d) {
            if (!event.active) force.alphaTarget(0.3).restart();
            d.fx = d.x;
            d.fy = d.y;

            PosX = d.x
            PosY = d.y
        }

        function dragged(event, d) {
            d.fx = event.x;
            d.fy = event.y;
        }

        function dragEnded(event, d) {
            if (!event.active) force.alphaTarget(0);
            d.fx = undefined;
            d.fy = undefined;
        }

    </script>
</body>

</html>


from D3.js highlight related nodes / links

Save in Contacts works in Android 10 but not in Android 11

Am facing some weird issue I have an android app that allows users to save the profile details in Contacts like Name, phone, email, this functionality is working in Android 10 but not in Android 11. In Android 11 am able to see the contacts saved in the Phone App contacts page but this not reflecting in Contacts App. Are there any extra permissions or settings we need in Android 11 to get this work?

enter image description here

Contact Saved in phone app contacts page

enter image description here

But not reflecting in the Contacts App



from Save in Contacts works in Android 10 but not in Android 11

SQLAlchemy: how to create a relationship programmatically

I'd like to create a 1:n relationship between two tables dynamically. My DB model is mapped via SQLAlchemy but due to some special features of my application I can not use the default declarative way.

E.g.

class Foo(Base):
    id = Column(Integer, autoincrement=True, primary_key=True)
    flag = Column(Boolean)

class Bar(Base):
    id = Column(Integer, autoincrement=True, primary_key=True)
    foo_id = Column(Integer, ForeignKey('foo.id'))
    # declarative version:
    # foo = relationship(Foo)

So I want to add relationship named "foo" to the mapped class "Bar" after Bar was defined and SQLAlchemy did its job of defining a mapper etc.

Update 2017-09-05: Why is this necessary for me? (I thought I could omit this because I think it mostly distracts from the actual problem to solve but since there were comments abouts it...)

First of all I don't have a single database but hundreds/thousands. Data in old databases must not be altered in any way but I want a single source code to access old data (even though data structure and calculation rules change significantly).

Currently we use multiple model definitions. Later definitions extend/modify previous ones. Often we manipulate SQLAlchemy models dynamically. We try not to have code in mapped classes because we think it will be much harder ensuring correctness of that code after changing a table many times (code must work in every intermediate step).

In many cases we extend tables (mapped classes) programatically in model X after it was initially defined in model X-1. Adding columns to an existing SQLAlchemy ORM class is manageable. Now we are adding a new reference column an existing table and a relationship() provides a nicer Python API.



from SQLAlchemy: how to create a relationship programmatically

Is there a way to tell manifest merger not to add path data to intent-filter

I'm using navigation component and navigation graph for handling deeplinks. I've added deeplinks to my destinations like below

<deepLink app:uri="example://foo" />

Intent Filter documentation mentions:

If a filter specifies a scheme and an authority but no path, all URIs with the same scheme and authority pass the filter, regardless of their paths.

According to this, a uri like example://foo should match my deeplink. But manifest merger adds this line to my intent filter, even if don't add any path to my uri:

<data android:path="/" />

So only a example://foo/ is matched. Is there a way to tell manifest merger or navigation graph to remove this data element from my intent filter?



from Is there a way to tell manifest merger not to add path data to intent-filter

call an async def when a cog is initialized

So what I want to do is call an async function when a cog is initialized,

I've came up with couple of ideas but none worked,
I mostly tried to call the async function with __init__ but i can't since it's considered a sync initializer.

for context this is main.py

import discord
import os
from discord.ext import commands



intents = discord.Intents.default()
intents.members = True
bot = commands.Bot(command_prefix=',', intents=intents)



for filename in os.listdir('./Cogs'):
    if filename.endswith('.py'):
        bot.load_extension(f'Cogs.{filename[:-3]}')
        print(filename)



@bot.event
async def on_ready():

    print(f'The bot has logged in as {bot.user}')
    print('''LOADING COGS:---''')



@bot.command()
async def load(ctx, extension):
    bot.load_extension(f'Cogs.{extension}')
    await ctx.send(f'loaded {extension}')

@bot.command()
async def reload(ctx, extension):
    bot.reload_extension(f'Cogs.{extension}')
    await ctx.send(f'reloaded {extension}')

@bot.command()
async def unload(ctx, extension):
    bot.unload_extension(f'Cogs.{extension}')
    await ctx.send(f'unloaded {extension}')





bot.run('TOKEN')

this is the closest I've come to a working code but it returns coroutine instead of None,
this is foo.py:

import discord
from discord.ext import commands
from main import bot


class variables(commands.Cog):

    async def _async_init_(self, bot):
        print('''async init''')
        self.bot = bot
        await variables.funcs_variables
        # return None # this does nothing as far as i understand

    __init__ = _async_init_



    global guild_id
    guild_id = # **guild's ID**



    @commands.command()
    async def funcs_variables():

        global guild
        # i want this to run once the cog is initialized
        guild = discord.utils.find(lambda g : g.id == guild_id, bot.guilds)



def setup(bot):
    bot.add_cog(variables(bot))
    

this is the error it throws:

d:\Discord bots\Discord bot - Cogs NEW\Cogs\foo.py:32: RuntimeWarning: coroutine 'variables._async_init_' was never awaited
  bot.add_cog(variables(bot))
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 618, in _load_from_module_spec
    setup(self)
  File "d:\Discord bots\Discord bot - Cogs NEW\Cogs\foo.py", line 32, in setup
    bot.add_cog(variables(bot))
TypeError: __init__() should return None, not 'coroutine'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 606, in _load_from_module_spec
    spec.loader.exec_module(lib)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "d:\Discord bots\Discord bot - Cogs NEW\Cogs\foo.py", line 3, in <module>
    from main import bot
  File "d:\Discord bots\Discord bot - Cogs NEW\main.py", line 15, in <module>
    bot.load_extension(f'Cogs.{filename[:-3]}')
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 678, in load_extension    
    self._load_from_module_spec(spec, name)
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 623, in _load_from_module_spec
    raise errors.ExtensionFailed(key, e) from e
discord.ext.commands.errors.ExtensionFailed: Extension 'Cogs.foo' raised an error: TypeError: __init__() should return None, not 'coroutine'
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\Discord bots\Discord bot - Cogs NEW\main.py", line 15, in <module>
    bot.load_extension(f'Cogs.{filename[:-3]}')
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 678, in load_extension    
    self._load_from_module_spec(spec, name)
  File "C:\Users\Bruh\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\ext\commands\bot.py", line 608, in _load_from_module_spec
    del sys.modules[key]
KeyError: 'Cogs.foo'

Edit1:

I've come accross this code which works nicely
But i've got another problem it's that bot comes back as a nontype

import discord
from discord.ext import commands
import inspect
# from main import bot
import asyncio

def setup(bot):
    print('''setup''')
    bot.add_cog(variables(bot))

class variables(commands.Cog):

    def __init__(self, bot):
        self.bot = bot
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        result = loop.run_until_complete(variables.funcs_variables(self))





    global guild_id
    guild_id = # guild's id

    @commands.command()
    async def funcs_variables(self):
        print('''Loading Variables''')
        #primaries
        global frame, guild
        frame = inspect.stack()[0][0]
        guild = discord.utils.find(lambda g : g.id == guild_id, self.bot.guilds)

        # commands like so will return the error: 'NoneType' object has no attribute 'attribute'
        await self.bot.change_presence(activity=discord.Streaming(name='''Author: Author''', url='https://www.youtube.com/watch?v=dQw4w9WgXcQ'))
        global special_role
        special_role = discord.utils.get(guild.roles, id='special_role id')

    @commands.command()
    async def test(self, ctx):
        await ctx.send(f'''{guild}''') #sends "None"
        print(guild)

edit2:

this is the working code:

main.py

import discord
import os
from discord.ext import commands



intents = discord.Intents.default()
intents.members = True
bot = commands.Bot(command_prefix=',', intents=intents)



async def load_cogs():

    for filename in os.listdir('./Cogs'):
        if filename.endswith('.py'):
            # await asyncio.sleep(1)
            bot.load_extension(f'Cogs.{filename[:-3]}')
            print(filename)



@bot.event
async def on_ready():

    #print in the console once it starts
    print(f'The bot has logged in as {bot.user}')
    print('''LOADING COGS:---''')

    await load_cogs()



@bot.command()
async def load(ctx, extension):
    bot.load_extension(f'Cogs.{extension}')
    await ctx.send(f'loaded {extension}')

@bot.command()
async def reload(ctx, extension):
    bot.reload_extension(f'Cogs.{extension}')
    await ctx.send(f'reloaded {extension}')

@bot.command()
async def unload(ctx, extension):
    bot.unload_extension(f'Cogs.{extension}')
    await ctx.send(f'unloaded {extension}')





bot.run('TOKEN')

and foo.py you can find it in the beautiful top answer



from call an async def when a cog is initialized

How to put a dynamic data from firestore in the function where() and also use the snap.size to count the total query to be passed in a graph?

I have this data from firestore and I wanted to retrieve it dynamically with a where() but this is the error I'm getting:

TypeError: vaccines is not a function

The user collection:

[![enter image description here][1]][1]

The vaccines collection:

[![enter image description here][2]][2]

Below are the codes:

 const Vaccine = () => {
      const [vaccines, setVaccines] = useState([]);
      useEffect(() => {
        const unsubscribe = firestore
          .collection("vaccines")
          .onSnapshot((snapshot) => {
            const arr = [];
            snapshot.forEach((doc) =>
              arr.push({
                ...doc.data(),
                id: doc.id,
              })
            );
            setVaccines(arr);
          });
    
        return () => {
          unsubscribe();
        };
      }, []);

  //this is the part where the error happens
  {
    vaccines &&
      vaccines((index) => {
        useEffect(async () => {
          const ref = firestore.collection("users");
          const snapshot = await ref
            .where("doses.selectedVaccine", "==", `${index.vaccine}`)
            .where("doses.dose1", "==", true)
            .where("doses.dose2", "==", true)
            .onSnapshot((snap) => {
              console.log("overall", snap.size);
            });
        }, []);
      });
  }
  return <div></div>;
};

export default Vaccine;

I tried this, but it says that there's an error: Error: Rendered more hooks than during the previous render. I wanted to loop through all of the vaccines so I could get the total number of users of those dose1 with true and dose2 with false.

  vaccines &&
    vaccines.map((v, index) => {
      useEffect(async () => {
        const ref = firestore.collection("users");
        const snapshot = await ref
          .where("doses.selectedVaccine", "==", `${index.vaccine}`)
          .where("doses.dose1", "==", true)
          .where("doses.dose2", "==", true)
          .onSnapshot((snap) => {
            setSize(snap.size);
          });
      }, []);
    });

Updated codes: Not much has changed: I wanted to get the snap.size of each of the vaccines so I can put it in the graph. The codes under the comment of dynamic data has en error that says:

error TypeError: doc.data(...).forEach is not a function



from How to put a dynamic data from firestore in the function where() and also use the snap.size to count the total query to be passed in a graph?

tkinter "scrolledtext" copy paste doesn't work reliably

I have found a behaviour which seems to be a bug in tkinter. If you run the following (minimal to reproduce the bug) code:

import tkinter, tkinter.simpledialog, tkinter.scrolledtext
root = tkinter.Tk('test')
text = tkinter.scrolledtext.ScrolledText(master=root, wrap='none')
text.pack(side="top", fill="both", expand=True, padx=0, pady=0)
text.insert(tkinter.END, 'abc\ndef\nghi\nijk')
root.mainloop()

then:

  • select one row in the scrolledtext widget, e.g. the row "ghi",
  • copy it with CTRL+C
  • do nothing else and close the app

Then paste it (CTRL+V) in any other Windows app: it won't work, nothing will be pasted. Why?

How to solve this?


Note: the expected behaviour is that text copied with CTRL+C should persist in the clipboard even if the app is closed. This is the default behaviour in many Windows software. Example here with notepad.exe: link to the animated screen capture: https://i.imgur.com/li7UvYw.mp4

enter image description here



from tkinter "scrolledtext" copy paste doesn't work reliably

Hierarchical clustering label based on their merging order in python

lets say, I have this type of Hierarchical clustering as below diagram. To get the clustering labels, I need to define proper threshold distance. For example, If I put the threshold at 0.32, I probably would get 3 clusters and if I set around 3.5, I would get 2 clusters from this below diagram.

Instead of using threshold and use some fixed distance, I would like to get the clustering label based on their merging orders.

I would like to define the clustering based on their merging; like first merging, second merging, etc.

For example, here I would like to get clustering labels, when they do at least first merge and that would be 3 clusters;

cluster1: p1
cluster2: p3 and p4
cluster3: p2 and p5.

If I set here, find the clustering when there is at least second merging happens. In this case, I would have 2 clusters such as:

cluster1: p1
cluster2 = p3, p4, p2 and p5.

Does scipy has builtin method to extract this kind of information. If not, is there any way that I can extract this type of information from hierarchical clustering ? Any suggestions would be great.

enter image description here



from Hierarchical clustering label based on their merging order in python

how to put data into Excel table with for loop pandas?

Following my previous question, now i'm trying to put data in a table and convert it to an excel file but i can't get the table i want, the excel file contains just the last hotel in my list if anyone can help or explain what's the cause of it, this is the final output i want to get

enter image description here

this the data i'm printing

Byzance Nabeul : Chambre Double - {'All Inclusive soft': ('208', '166', '25%'), 'Demi Pension': 138}
Palmyra Club Nabeul Nabeul : Double Standard - {'All Inclusive soft': ('225', '180', '25%')}

and here is my code

#!/usr/bin/env python
# coding: utf-8
import time
from time import sleep
import ast
import xlwt
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.common.exceptions import StaleElementReferenceException, NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import pandas as pd


driver = webdriver.Chrome("C:\\Users\\marketing2\\Documents\\chromedriver.exe")
driver.get('https://tn.tunisiebooking.com/')

def exists(xpath):
        try:
            driver.find_element_by_id(xpath);
        except NoSuchElementException:
            return "false"
        else:
            return "true"

# params to select
params = {
    'destination': 'Nabeul',
    'date_from': '11/09/2021',
    'date_to': '12/09/2021',
    'bedroom': '1'
}

# select destination
destination_select = Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, 'ville_des'))))
destination_select.select_by_value(params['destination'])

# select bedroom
bedroom_select = Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'select_ch'))))
bedroom_select.select_by_value(params['bedroom'])

# select dates
script = f"document.getElementById('checkin').value ='{params['date_from']}';"
script += f"document.getElementById('checkout').value ='{params['date_to']}';"
script +=  f"document.getElementById('depart').value ='{params['date_from']}';"
script += f"document.getElementById('arrivee').value ='{params['date_to']}';"
driver.execute_script(script)

# submit form
btn_rechercher = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="boutonr"]')))
btn_rechercher.click()
    
#----------   
if (exists('plus_res')=="true"):
   
    btn_plus = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'plus_res')))
    btn_plus.click()
    sleep(10)
else :
    pass
    
urls = []    
records = []
hotels = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//div[starts-with(@id,'produit_affair')]")))

for hotel in hotels:
    link = hotel.find_element_by_xpath(".//span[@class='tittre_hotel']/a").get_attribute("href")
    urls.append(link)

for url in urls:
    driver.get(url)
       
    def existsElement(xpath):
        try:
            driver.find_element_by_id(xpath);
        except NoSuchElementException:
            return "false"
        else:
            return "true"
   
    if (existsElement('result_par_arrangement')=="false"):
   
        btn_t = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="moteur_rech"]/form/div/div[3]/div')))

        btn_t.click()
        sleep(10)
    else :
        pass
               
    
    try:
        name = str(WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//div[@class='bloc_titre_hotels']/h2"))).text)
        arropt = driver.find_element_by_xpath("//div[contains(@class,'line_result')][1]")
        opt = str(arropt.find_element_by_tag_name("b").text)
        num = len(arropt.find_elements_by_tag_name("option"))
        optiondata = {}
        achats = {}
        marges= {}
        selection = Select(driver.find_element_by_id("arrangement"))

        for i in range(num):
            try:
                selection = Select(driver.find_element_by_id("arrangement"))
                selection.select_by_index(i)
                time.sleep(2)

                arr = driver.find_element_by_xpath("//select[@id='arrangement']/option[@selected='selected']").text
                prize = driver.find_element_by_id("prix_total").text

                optiondata[arr] = (int(prize))

                btn_passe = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="resultat"]/div/form/div/div[2]/div[1]/div[2]/div[2]/div')))
                btn_passe.click()



                # params to select
                params = {
                            'civilite_acheteur': 'Mlle',
                            'prenom_acheteur': 'test',
                            'nom_acheteur': 'test',
                            'e_mail_acheteur': 'test@gmail.com',
                            'portable_acheteur': '22222222',
                            'ville_acheteur': 'Test',
                        }

                # select civilite
                civilite_acheteur = Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.NAME, 'civilite_acheteur'))))
                civilite_acheteur.select_by_value(params['civilite_acheteur'])

                # saisir prenom 
                script  = f"document.getElementsByName('prenom_acheteur')[0].value ='{params['prenom_acheteur']}';"
                script += f"document.getElementsByName('nom_acheteur')[0].value ='{params['nom_acheteur']}';"
                script += f"document.getElementsByName('e_mail_acheteur')[0].value ='{params['e_mail_acheteur']}';"
                script += f"document.getElementsByName('portable_acheteur')[0].value ='{params['portable_acheteur']}';"
                script += f"document.getElementsByName('ville_acheteur')[0].value ='{params['ville_acheteur']}';"
                driver.execute_script(script)

                # submit form
                btn_agence = driver.find_element_by_id('titre_Nabeul')
                btn_agence.click()

                btn_continuez = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'boutonr')))
                btn_continuez.click()

                achat = str(int(driver.find_element_by_xpath('/html/body/header/div[2]/div[1]/div[1]/div[4]/div[2]/div[2]').text.replace(' TND', '')))

                achats[arr]=achat

                marge =str(int(((float(prize) - float(achat)) / float(achat)) * 100))+"%";
                marges[arr]=marge
                optiondata[arr]=prize,achat,marge
                
                
                driver.get(url)
                btn_display = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="moteur_rech"]/form/div/div[3]/div')))

                btn_display.click()
                sleep(10)
               

            except StaleElementReferenceException:
                pass

            

    except NoSuchElementException:
        pass
    

    s="{} : {} - {}".format(name, opt, optiondata)
    

    ds = []

    for l in s.splitlines():
            d = l.split("-")

            if len(d) > 1:
                df = pd.DataFrame(ast.literal_eval(d[1].strip()))
                ds.append(df)
    for df in ds:
            df.reset_index(drop=True, inplace=True)

    df = pd.concat(ds, axis= 1)

    cols = df.columns

    cols = [((col.split('.')[0], col)) for col in df.columns]

    df.columns=pd.MultiIndex.from_tuples(cols)

    print(df.T)
    df.to_excel("v.xlsx")

it's displaying "Palmyra club Nabeul Nabeul" only How can i solve the probleme please this the final and most important part and thank you in advance.



from how to put data into Excel table with for loop pandas?

Why do I get an illegal memory access when I'm calling a kernel in pycuda?

I'm trying to implement a neuron model with Hodgkin and Huxley formalism on my RTX 2080 Ti with PyCuda. The code is quite large so I wont put all of it here. the first part of my class is to set the number of neurons, create all variables in the GPU and get the block and grid size according on the number of neurons (1 neuron by thread)

class Inter_PC:
    def __init__(self, ):

        self.NbODEs = 25
        self.NbCells = int(1024 * 1)
        self.init_vector()
        self.init_vector_param()
        self.Create_GPU_SourceModule()

        BLOCK_SIZE = 1024
        self.grid = (int(self.NbCells / BLOCK_SIZE), 1)
        self.block = (BLOCK_SIZE, 1, 1)

In the function init_vector and init_vector_param, I put vectors to compute ODE results in the GPU

def init_vector(self):
    self.Vs_PC_dydx1 = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    self.Vs_PC_dydx2 = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    self.Vs_PC_dydx3 = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    self.Vs_PC_dydx4 = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    self.Vs_PC_y = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    self.Vs_PC_yt = self.put_vect_on_GPU(np.zeros((self.NbCells), dtype=np.float32))
    ...
def init_vector_param(self):
    self.E_leak = self.put_vect_on_GPU(np.ones((self.NbCells), dtype=np.float32) * -65)
    self.E_Na = self.put_vect_on_GPU(np.ones((self.NbCells), dtype=np.float32) * 55)
    ...
def put_vect_on_GPU(self, Variable):
    Variable_gpu = cuda.mem_alloc(Variable.nbytes)
    cuda.memcpy_htod(Variable_gpu, Variable)
    return Variable_gpu

In the function Create_GPU_SourceModule, I create kernels to use on the GPU.

def Create_GPU_SourceModule(self):
    self.mod = SourceModule("""
    #include <math.h>
      __global__ void m_inf_PC(float *V_PC, float *res)
      {
        int idx = threadIdx.x + blockDim.x * blockIdx.x; 
        res[idx] = 1.0 / ( 1. * exp(-(V_PC[idx] + 40.) / 3.));
      }
      __global__ void h_inf_PC(float *V_PC, float *res)
      {
        int idx = threadIdx.x + blockDim.x * blockIdx.x; 
        res[idx] = 1.0 / ( 1. * exp((V_PC[idx] + 45.) / 3.)); 
      }
      ...

I have the a function to update all my variables in a RK4 solver updateParameters

def setParameters(self):
    func = self.mod.get_function("set_vect_val")
    func(self.Vs_PC_y, self.E_leak, block=self.block, grid=self.grid)
    func = self.mod.get_function("set_vect_val")
    func(self.Vd_PC_y, self.E_leak, block=self.block, grid=self.grid)
    func = self.mod.get_function("h_inf_PC")
    func(self.Vs_PC_y, self.h_s_PC_y, block=self.block, grid=self.grid)
    func = self.mod.get_function("m_KDR_inf_PC")
    func(self.Vs_PC_y, self.m_KDR_s_PC_y, block=self.block, grid=self.grid)
    func = self.mod.get_function("m_m_inf_PC")
    func(self.Vs_PC_y, self.m_s_PC_y, block=self.block, grid=self.grid)
    func = self.mod.get_function("m_m_inf_PC")
    func(self.Vd_PC_y, self.m_d_PC_y, block=self.block, grid=self.grid)
    func = self.mod.get_function("h_inf_PC")
    func(self.Vd_PC_y, self.h_d_PC_y, block=self.block, grid=self.grid)

When I run the code I get this error:

Traceback (most recent call last):
 File "C:/Users/maxime/Desktop/SESAME/PycharmProjects/Modele_Micro3/Class_PyrCell_GPU.py", line 1668, in <module>
    Vm = PC.rk4_Time(30000)
 File "C:/Users/maxime/Desktop/SESAME/PycharmProjects/Modele_Micro3/Class_PyrCell_GPU.py", line 1637, in rk4_Time
    self.updateParameters()
  File "C:/Users/maxime/Desktop/SESAME/PycharmProjects/Modele_Micro3/Class_PyrCell_GPU.py", line 998, in updateParameters
    func = self.mod.get_function("h_inf_PC")
  File "C:\Python389\lib\site-packages\pycuda\compiler.py", line 326, in get_function
    return self.module.get_function(name)
pycuda._driver.LogicError: cuModuleGetFunction failed: an illegal memory access was encountered
PyCUDA WARNING: a clean-up operation failed (dead context maybe?)
cuMemFree failed: an illegal memory access was encountered

What I'm not understanding is that the error does not occur the first time I use the kernel h_inf_PC, it happens on the 13th line of the function setParameters but I already calling the same kernel in line 5 of the same function. If I comment out the calling to the kernel (h_inf_PC) that causes the issue, the error switched on another calling to a kernel but not necessarily the next one.



from Why do I get an illegal memory access when I'm calling a kernel in pycuda?

Saving an animated Matplotlib graph as a GIF file results in a different looking plot

I created an animated plot using FuncAnimation from the Matplotlib Animation class, and I want to save it as a .gif file. When I run the script, the output looks normal, and looks like this (the animation works fine):

enter image description here

However, when I try to save the animated plot as a .gif file using ImageMagick or PillowWriter, the plot looks like the following graph:

enter image description here

The lines are clearly much thicker, and in general, just looks very bad. The problem is attributed to the points (the purple, and red circles). Thus it seems like the plot is writing over each frame (which I think is the case). I can avoid this by just getting rid of them all together. But I don't want to do that as it would be hard to see the lines.

Here is the code:

line, = ax.plot([], [], color = 'blue', lw=1)
line2, = ax.plot([], [], color = 'red', lw=1)
line3, = ax.plot([], [], color = 'purple', lw=1)
def animate(i):
    line.set_data(x1[:i], y1[:i])
    line2.set_data(x2[:i], y2[:i])
    line3.set_data(x3[:i], y3[:i])
    point1, = ax.plot(x1[i], y1[i], marker='.', color='blue')
    point2, = ax.plot(x2[i], y2[i], marker='.', color='red')
    point3, = ax.plot(x3[i], y3[i], marker='.', color='purple')
    return line, line2, line3, point1, point2, point3,
        
ani = animation.FuncAnimation(fig, animate, interval=20, blit=True, repeat=False, frames=1000, save_count=1000)    
ani.save("TLI.gif", writer='imagemagick',fps=60)

The arrays x1, y1, x2, y2, x3, y3 are all 1D arrays that contain the x, y coordinates. So why is this happening? Why is it that the .gif file doesn't show what the plot shows when I run it directly? And also, how can I fix this?

I am also aware of this Stack Overflow question: matplotlib animation save is not obeying blit=True but it seems to work just fine in plt.show() which means the problem is definitely attributed to blitting. However, reading the answer of that question did not solve my problem because that only refers to ax.text opposed to a regular point plotted via ax.plot.



from Saving an animated Matplotlib graph as a GIF file results in a different looking plot

Post image to Instagram from a Javascript using Instagram API

Instagram Graph API:
https://developers.facebook.com/docs/instagram-api/

Content Publishing:
https://developers.facebook.com/docs/instagram-api/guides/content-publishing/

My code Javascript in Google App Script:

function InstagramPost() {

  const id = '123456789';
  const image = 'https://www.w3schools.com/images/w3schools_green.jpg';
  const text = 'Hello%20World';
  const access_token = 'TESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTESTTEST';
  const container = 'https://graph.facebook.com/v11.0/' + id + '/media?image_url=' + image + '&caption=' + text + '&access_token=' + access_token;

  const response = UrlFetchApp.fetch(container);
  const creation = response.getContentText();

  Logger.log(creation);
}

The return in my Logger of my container to Post via Instagram API request comes as follows:

{
  "data": [
    {
      "id": "11111111111111111"
    },
    {
      "id": "22222222222222222"
    },
    {
      "id": "33333333333333333"
    },
    {
      "id": "44444444444444444"
    },
    {
      "id": "55555555555555555"
    },
    {
      "id": "66666666666666666"
    },
    {
      "id": "77777777777777777"
    },
    {
      "id": "88888888888888888"
    },
    {
      "id": "99999999999999999"
    },
    {
      "id": "00000000000000000"
    },
    {
      "id": "AAAAAAAAAAAAAAAAA"
    },
    {
      "id": "BBBBBBBBBBBBBBBBB"
    },
    {
      "id": "CCCCCCCCCCCCCCCCC"
    },
    {
      "id": "DDDDDDDDDDDDDDDDD"
    },
    {
      "id": "EEEEEEEEEEEEEEEEE"
    },
    {
      "id": "FFFFFFFFFFFFFFFFF"
    },
    {
      "id": "GGGGGGGGGGGGGGGGG"
    },
    {
      "id": "HHHHHHHHHHHHHHHHH"
    },
    {
      "id": "IIIIIIIIIIIIIIIII"
    },
    {
      "id": "JJJJJJJJJJJJJJJJJ"
    },
    {
      "id": "KKKKKKKKKKKKKKKKK"
    },
    {
      "id": "LLLLLLLLLLLLLLLLL"
    },
    {
      "id": "MMMMMMMMMMMMMMMMM"
    },
    {
      "id": "NNNNNNNNNNNNNNNNN"
    },
    {
      "id": "OOOOOOOOOOOOOOOOO"
    }
  ],
  "paging": {
    "cursors": {
      "before": "QWOURQWNGEWRONHWENYWPETGNWQPGNPGNWEPGNWEPGNWEPNGWPENGPWEG",
      "after": "WIWEPGNEPBNWE´GNÉ´BNWE´BNWÉBWNEB´WENBNWEBWEBEWBWE"
    },
    "next": "https://graph.facebook.com/v11.0/11111111111111111/media?access_token=PQWNFWPQINPWNBQPWNBQPWNBPQWNVQWPNVPQWVNPQWPVNQPWNVQPWVNQPWNVPQWNVQPWNVQPWVNQASASLGÇAJKSGLJAAÇSNAÇKNSVÇLKNASBÇANSBÇAS"
  }
}

To make the final call for post image it is necessary to use an creation_id=:

const sendinstagram = 'https://graph.facebook.com/v11.0/' + id + '/media_publish?creation_id=' + creation + '&access_token=' + access_token;
UrlFetchApp.fetch(sendinstagram);

If the return from the container is several id in sequence, how do I know which one to define for the call?

Note: I can't loop to try every id because Instagram has a daily limit of 25 calls and posts, so if I did that I would end up with my calls just trying to post a single image.

If you have knowledge about the Instagram API and can help me with an additional one, I would appreciate it. That would be in regards to how I can do to automatically update the token, because it expires in a few hours and I would like to update it automatically.



from Post image to Instagram from a Javascript using Instagram API

preventDefault, stopPropagation, and cancelBubble not enough to prevent contextMenu from trying to open in Firefox mobile test view

If you open the test URL in Firefox desktop version 91.0.2 (Latest at the time) Windows 10 64-bit (Probably other versions too) and open the F12 menu, then click the Responsive Design Mode button (ctrl+shift+m... Aka the mobile view button) and hold left click to the left of the image, you'll see the number go up to about 25 - then stop and reset to zero, and trigger onCancel. My theory is that's how long it takes until the context menu is normally triggered.

If you try it in regular desktop view or Chrome's regular desktop view - it works as expected - it counts up to 100 (And beyond) until you let go, then onFinish gets triggered.

The full code is here: Code sandbox

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1, shrink-to-fit=no"
    />
    <meta name="theme-color" content="#000000" />
    <!--
      manifest.json provides metadata used when your web app is added to the
      homescreen on Android. See https://developers.google.com/web/fundamentals/engage-and-retain/web-app-manifest/
    -->
    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
    <link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico" />
    <!--
      Notice the use of %PUBLIC_URL% in the tags above.
      It will be replaced with the URL of the `public` folder during the build.
      Only files inside the `public` folder can be referenced from the HTML.

      Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
      work correctly both with client-side routing and a non-root public URL.
      Learn how to configure a non-root public URL by running `npm run build`.
    -->
    <title>React App</title>
    <script>
      window.addEventListener(
        "contextmenu",
        function (e) {
          // do something here...
          e.preventDefault && e.preventDefault();
          e.stopPropagation && e.stopPropagation();
          e.cancelBubble = true;
          return false;
        },
        false
      );

      const noContext = document.getElementById("ItemImage");

      noContext.addEventListener("contextmenu", (e) => {
        e.preventDefault && e.preventDefault();
        e.stopPropagation && e.stopPropagation();
        e.cancelBubble = true;
        return false;
      });

      function absorbEvent_(event) {
        var e = event;
        e.preventDefault && e.preventDefault();
        e.stopPropagation && e.stopPropagation();
        e.cancelBubble = true;
        return false;
      }

      function preventLongPressMenu(node) {
        node.ontouchstart = absorbEvent_;
        node.ontouchmove = absorbEvent_;
        node.ontouchend = absorbEvent_;
        node.ontouchcancel = absorbEvent_;
      }

      function init() {
        preventLongPressMenu(document.getElementById("ItemImage"));
      }
    </script>
    <style>
      /* https://www.arungudelli.com/tutorial/css/disable-text-selection-in-html-using-user-select-css-property/ */
      .disable-select {
        user-select: none; /* supported by Chrome and Opera */
        -webkit-user-select: none; /* Safari */
        -khtml-user-select: none; /* Konqueror HTML */
        -moz-user-select: none; /* Firefox */
        -ms-user-select: none; /* Internet Explorer/Edge */
      }
    </style>
  </head>

  <body>
    <noscript>
      You need to enable JavaScript to run this app.
    </noscript>
    <div id="root"></div>
    <!--
      This HTML file is a template.
      If you open it directly in the browser, you will see an empty page.

      You can add webfonts, meta tags, or analytics to this file.
      The build step will place the bundled scripts into the <body> tag.

      To begin the development, run `npm start` or `yarn start`.
      To create a production bundle, use `npm run build` or `yarn build`.
    -->
  </body>
</html>

App.js

import React, { useCallback, useRef, useState } from "react";

import "./styles.css";
import { useLongPress } from "use-long-press";
import Item from "./Item";

function App() {
  const [progress, setProgress] = useState(0);
  const progressTimer = useRef();
  function handleTime() {
    setProgress((prevState) => (prevState += 5));
  }

  const callback = useCallback((event) => {
    event.preventDefault && event.preventDefault();
    event.stopPropagation && event.stopPropagation();
    event.cancelBubble = true;
    console.log("long pressed!");
  }, []);

  const longPressEvent = useLongPress(callback, {
    onStart: (event) => {
      console.log("On Start");
      event.preventDefault && event.preventDefault();
      event.stopPropagation && event.stopPropagation();
      event.cancelBubble = true;
      progressTimer.current = setInterval(handleTime, 100);
    },
    onFinish: (event) => {
      console.log("On finish");
      event.preventDefault && event.preventDefault();
      event.stopPropagation && event.stopPropagation();
      event.cancelBubble = true;
      setProgress(0);
      clearInterval(progressTimer.current);
    },
    onCancel: (event) => {
      console.log("On cancel");
      event.preventDefault && event.preventDefault();
      event.stopPropagation && event.stopPropagation();
      event.cancelBubble = true;
      setProgress(0);
      clearInterval(progressTimer.current);
    },
    threshold: 2000,
    captureEvent: true,
    cancelOnMovement: false,
    detect: "both"
  });

  let content = (
    <div className="content-center">
      {progress}
      <Item
        events={longPressEvent}
        name="name"
        image="file.png"
        progress={progress}
      />
    </div>
  );

  return <React.Fragment>{content}</React.Fragment>;
}

export default App;

Item.js

import React from "react";
import "./Item.css";
import VerticalProgress from "./VerticalProgress";
import faker from "faker";

export default function Item(props) {
  return (
    <div className="flex justify-center w-full disable-select">
      <div
        className="float-left absolute z-50 w-full disable-select"
        style=
        {...props.events}
      >
        <div></div>
      </div>
      <VerticalProgress className="z-0" progress={props.progress} />
      <img
        className="z-0 disable-select"
        src={faker.image.image()}
        alt={props.name}
        height="200"
        id="ItemImage"
      />
    </div>
  );
}

My question is: How can I prevent onCancel from being called early in Firefox desktop mobile test mode? If it's happening there, it's bound to be happening in other browsers like Safari or actual Firefox mobile. Even if it isn't, this is a very much unintended side effect and is making testing mobile difficult.

Second question: What's causing the

IndexSizeError: Selection.getRangeAt: 0 is out of range

Errors?



from preventDefault, stopPropagation, and cancelBubble not enough to prevent contextMenu from trying to open in Firefox mobile test view

Unable to get response from FreshDesk API in proxy mode {"code" : "invalid_credentials", "message" : "You need to be logged in to perform action"}

So I'm using FreshDesk API and able to get the response using request module, But whenever I'm using proxy servers I'm unable to get the response.

import base64
import requests
import os
from requests.auth import HTTPBasicAuth
import ssl
method = "get"
url = "https://mycompanydomain.freshdesk.com/api/v2/tickets"
apiKey = "XXXXXXXXX"
secret = "x"
os.environ["REQUESTS_CA_BUNDLE"] = "Path to CA Certs"
auth = HTTPBasicAuth(apiKey, secret)
rsp = requests.request(method, url, headers=None, auth=auth)
print(rsp.text)

But whenever I'm using the proxy server in my organization, I'm getting an error message as {"code":"invalid_credentials","message":"You have to be logged in to perform this action."}

Code which I'm using for the proxy servers

import base64
import requests
import http.client
import urllib.parse
method = "get"
apiKey = "XXXXXXXX"
secret = "x"
url = "https://mycompanydomain.freshdesk.com/api/v2/tickets"
cred= '{}:{}'.format(apiKey, secret)
cred =  base64.b64encode(cred.encode('utf-8')).decode('utf-8')
authorization_headers = {
        'Proxy-Authorization': 'Basic {}'.format(cred)
}
conn = http.client.HTTPSConnection("11.125.250.121", 3128)
conn.set_tunnel("mycompanydomain.freshdesk.com", headers = authorization_headers)
headers = { 'Content-Type' : 'application/json' }
conn.request("GET", "/api/v2/tickets",headers = headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))

FreshDesk API Docs for using their API

curl -v -u abcdefghij1234567890:X -H "Content-Type: application/json" -X GET 'https://domain.freshdesk.com/api/v2/tickets'

Any possible way to resolve this error?



from Unable to get response from FreshDesk API in proxy mode {"code" : "invalid_credentials", "message" : "You need to be logged in to perform action"}

Monday 30 August 2021

How can I configure my python package to copy files to a subfolder of scripts post installation

I have a python package that comes with a variety of scripts in other languages. Setuptools already supports copying these scripts into the scripts directory of the environment to make them accessible from the command line. For this purpose I can simply use the following keyword in the setup commmand:

setup(
    ...
    scripts=["bin/script1.bat", "bin/script2.bat"],
    ...
    )

After installing the package, the script files will end up correctly in the scripts folder of the environment.

My question: Is there a way to have these files end up in a subfolder of the scripts directory? Something like scripts/odd_scripts/script1.bat and scripts/even_scripts/script2.bat.



from How can I configure my python package to copy files to a subfolder of scripts post installation

File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module, ModuleNotFoundError: No module named 'x'

EDIT

I'm trying to import algosec.models in a file inside the algobot package.

I've tried to add --hidden-import algosec, I've also tried to add the path before importing, using sys.path.append(./../algosec) this is the error message I get when I try to run the program:

Traceback (most recent call last):
  File "algobot_packer/algobot.py", line 2, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/cli/cli.py", line 3, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/microsoft_teams/mainloop.py", line 9, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/framework/configuration.py", line 34, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/framework/commands.py", line 22, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/framework/bot.py", line 4, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "algobot/framework/responses.py", line 9, in <module>
ModuleNotFoundError: No module named 'algosec'

the folder structure is:

  • algobot
    • algobot
    • algosec
    • algobot-packer
    • pyucwa

I'm using pyinstaller version 4.2 I didn't make any change in the code since the last time my executable file ran perfectly fine, but now I'm getting this error every time. the thing is - the folder 'algosec' is a subdirectory in my project, and it is noted in the pipfile and again, I didn't make any change in a while, and tested it recently (last tested in July 8th)), therefore I believe that it's a dependency issue but not sure which or how to solve.

I've tried some multiple changes that somehow worked on one run but when i tried to make these changes again it failed on other builds..



from File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module, ModuleNotFoundError: No module named 'x'

Amplify Federated Sign In not returning Email for Facebook

I'm using AWS Amplify to add social signing. With google I'm getting the email of the user but it's missing in facebook.

This is my code:

  federatedSignIn(provider: any): void {
    switch (provider) {
      case 'facebook':
          console.log("Authenticating using Facebook");
          Auth.federatedSignIn({ provider: CognitoHostedUIIdentityProvider.Facebook });
          break;
      case 'google':
        console.log("Authenticating using Google");
        Auth.federatedSignIn({ provider: CognitoHostedUIIdentityProvider.Google });
        break;
    }
     
  }

This is what the "Pre sign-up" Lambda sees as an event.

{
    "version": "1",
    "region": "us-east-1",
    "userPoolId": "us-east-xxxxxx",
    "userName": "Facebook_123456",
    "callerContext": {
        "awsSdkVersion": "aws-sdk-unknown-unknown",
        "clientId": "123456abcd"
    },
    "triggerSource": "PreSignUp_ExternalProvider",
    "request": {
        "userAttributes": {
            "email_verified": "false",
            "cognito:email_alias": "",
            "phone_number_verified": "false",
            "cognito:phone_number_alias": ""
        },
        "validationData": {}
    },
    "response": {
        "autoConfirmUser": false,
        "autoVerifyEmail": false,
        "autoVerifyPhone": false
    }
}

Userpool settings enter image description here

enter image description here

Attribute mapping enter image description here



from Amplify Federated Sign In not returning Email for Facebook

How to handle bulk create with multiple related models?

Here I have a form that have multiple input values with same name. I want to bulk create objects based on the following template design.

I think the current approach with zipping the list wouldn't work properly if one of the list are unequal.

What will be the better approach ? The front part should be like this as I posted you can check code snippet

   <script>
  $(document).on("click", ".q-name", function () {
    $(this).closest(".untitled").hide();
    $(this).siblings('.q-title').prop('hidden', false);
  });
</script>
<script>
  $(document).on("click", ".addOption", function () {
    option = `<div class="item">
                  <div>
                    <input
                      class="form-control"
                      type="text"
                      name="title"
                      placeholder="Enter title"
                    />
                  </div>
                  <div>
                    <select name="type" class="form-control">
                      Select
                      <option disabled>Select</option>
                      <option value="1">Option1</option>
                      <option value="2">Option2</option>
                    </select>
                  </div>`;
    $(this).closest(".options").prepend(option);
  });

  $(document).on("click", ".newOptionGroup", function () {
    group = `<input type="text" name="q_title" placeholder="model A field" class="form-control q-title"/></div>
              <p>Options of that model (Another model fields)</p>
              <div class="options">
                <div class="item">
                  <div>
                    <input
                      class="form-control"
                      type="text"
                      name="title"
                      placeholder="Enter title"/>
                  </div>
                  <div>
                    <select name="type" class="form-control">
                      Select
                      <option disabled>Select</option>
                      <option value="1">Option1</option>
                      <option value="2">Option2</option>

                    </select>
                  </div>
                </div>

                <div class="last">
                  <button type="button" class="btn btn-icon-only addOption">
                    Add more
                  </button>
                  <div>
                    <div class="custom-control custom-switch">
                      <input
                        name="is_document"
                        type="checkbox"
                        class="custom-control-input"
                        id="customSwitche"
                        value="1"
                      />
                      <label class="custom-control-label" for="customSwitche"
                        >Is File</label
                      >
                    </div>
                  </div>

                  <div></div>
                </div>
              </div>
            </div>
            <div class="option-group-new newOptionGroup">
              <button> Add New group</button>
            </div>
          </div>
          <div class="text-right mt-4">
            <button type="submit" class="btn btn-outline-grey">Submit</button>
          </div>`;

    $(".group-form").append(group);
});
</script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>

     <form class="group-form" method="post"> 
                <input
                  type="text"
                  name="q_title"
                  class="form-control q-title"
                  placeholder="model A field name"
                />
              <p>Options of that model (Another model fields)</p>
              </div>
              <div class="options">
                <div class="item">
                  <div>
                    <input
                      class="form-control"
                      type="text"
                      name="title"
                      placeholder="Enter title"
                    />
                  </div>
                  <div>
                    <select name="type" class="form-control">
                      Select
                      <option disabled>Select</option>
                     
                      <option value="1">Option1</option>
                      <option value="2">Option2</option>

                    </select>
                  </div>

                
                </div>

                <div class="last">
                  <button type="button" class="btn btn-icon-only addOption">
                    Add more
                  </button>
                  <div>
                    <div class="custom-control custom-switch">
                      <input
                        name="is_document"
                        type="checkbox"
                        class="custom-control-input"
                        id="customSwitche"
                        value="1"
                      />
                      <label class="custom-control-label" for="customSwitche"
                        >Is File</label
                      >
                    </div>
                  </div>

                  <div></div>
                </div>
              </div>
            </div>
            <div class="option-group-new newOptionGroup">
              <button type="button"> New group</button>
            </div>
          </div>
          <div class="text-right mt-4">
            <button type="submit" class="btn btn-outline-grey">Submit</button>
        
          </div>
        </form>
       </div>

Django views

    ques = Question.objects.get(id=kwargs["q_id"])
    q_title = request.POST.getlist("q_title")
    title = request.POST.getlist("title")
    types = request.POST.getlist("stype")
    is_file = request.POST.getlist("is_file", [0])
    params = zip(q_title, is_file, title, types)
    
    for p in params:

            q = Question.objects.create(
                title=p[0],
                is_file=p[1],
            )
            Option.objects.create(title=p[2], field_type=p[3], question=q)

EDIT:

The question titles and Option titles will be unequal since question can have unlimited options.

For example:

questions = ['q1', 'q2']
options = ['q1_option1', 'q1_option2', 'q2_option1', 'q2_option2', 'q2_option3']

I am not being able to track which option will belongs to particular question.

EDIT2:

ques_titles = ['q1-title', 'q2_title']
is_file = [True, False]

# ques_titles and is_file will be equal.

option_titles = ['q1_option1', 'q1_option2', 'q2-option1', 'q2-option2', 'q3-option3']
types = ['option1_type', 'option2_type', 'option3-type', 'option4-type', 'option5-type3'] 

 #option_titles and types list will be equal


from How to handle bulk create with multiple related models?

Wallpaper ad with DFP

We are trying to implement wallpaper add with DPF. We have followed https://dfphelp.in/helptopics/traffic-wallpaper-ad-with-click-function/ but no success, may be it is outdated.

Can any one help with this?



from Wallpaper ad with DFP

Scrapy Splash, How to deal with onclick?

I'm trying to scrape the following site

I'm able to receive a response but i don't know how can i access the inner data of the below items in order to scrape it:

I noticed that accessing the items is actually handled by JavaScript and also the pagination.

What should i do in such case?

enter image description here

Below is my code:

import scrapy
from scrapy_splash import SplashRequest


class NmpaSpider(scrapy.Spider):
    name = 'nmpa'
    http_user = 'hidden' # as am using Cloud Splash
    allowed_domains = ['nmpa.gov.cn']

    def start_requests(self):
        yield SplashRequest('http://app1.nmpa.gov.cn/data_nmpa/face3/base.jsp?tableId=27&tableName=TABLE27&title=%E8%BF%9B%E5%8F%A3%E5%8C%BB%E7%96%97%E5%99%A8%E6%A2%B0%E4%BA%A7%E5%93%81%EF%BC%88%E6%B3%A8%E5%86%8C&bcId=152904442584853439006654836900', args={
            'wait': 5}
        )

    def parse(self, response):
        goal = response.xpath("//*[@id='content']//a/@href").getall()
        print(goal)


from Scrapy Splash, How to deal with onclick?

Why does this gRPC call from the Google Secret Manager API hang when run by Apache?

In short:

I have a Django application being served up by Apache on a Google Compute Engine VM.

I want to access a secret from Google Secret Manager in my Python code (when the Django app is initialising).

When I do 'python manage.py runserver', the secret is successfully retrieved. However, when I get Apache to run my application, it hangs when it sends a request to the secret manager.

Too much detail:

I followed the answer to this question GCP VM Instance is not able to access secrets from Secret Manager despite of appropriate Roles. I have created a service account (not the default), and have given it the 'cloud-platform' scope. I also gave it the 'Secret Manager Admin' role in the web console.

After initially running into trouble, I downloaded the a json key for the service account from the web console, and set the GOOGLE_APPLICATION_CREDENTIALS env-var to point to it.

When I run the django server directly on the VM, everything works fine. When I let Apache run the application, I can see from the logs that the service account credential json is loaded successfully.

However, when I make my first API call, via google.cloud.secretmanager.SecretManagerServiceClient.list_secret_versions , the application hangs. I don't even get a 500 error in my browser, just an eternal loading icon. I traced the execution as far as:

grpc._channel._UnaryUnaryMultiCallable._blocking, line 926 : 'call = self._channel.segregated_call(...'

It never gets past that line. I couldn't figure out where that call goes so I couldnt inspect it any further than that.

Thoughts

I don't understand GCP service accounts / API access very well. I can't understand why this difference is occurring between the django dev server and apache, given that they're both using the same service account credentials from json. I'm also surprised that the application just hangs in the google library rather than throwing an exception. There's even a timeout option when sending a request, but changing this doesn't make any difference.

I wonder if it's somehow related to the fact that I'm running the django server under my own account, but apache is using whatever user account it uses?

Update

I tried changing the user/group that apache runs as to match my own. No change.

I enabled logging for gRPC itself. There is a clear difference between when I run with apache vs the django dev server.

On Django:

secure_channel_create.cc:178] grpc_secure_channel_create(creds=0x17cfda0, target=secretmanager.googleapis.com:443, args=0x7fe254620f20, reserved=(nil))
init.cc:167]                grpc_init(void)
client_channel.cc:1099]     chand=0x2299b88: creating client_channel for channel stack 0x2299b18
...
timer_manager.cc:188]       sleep for a 1001 milliseconds
...
client_channel.cc:1879]     chand=0x2299b88 calld=0x229e440: created call
...
call.cc:1980]               grpc_call_start_batch(call=0x229daa0, ops=0x20cfe70, nops=6, tag=0x7fe25463c680, reserved=(nil))
call.cc:1573]               ops[0]: SEND_INITIAL_METADATA...
call.cc:1573]               ops[1]: SEND_MESSAGE ptr=0x21f7a20
...

So, a channel is created, then a call is created, and then we see gRPC start to execute the operations for that call (as far as I read it).

On Apache:

secure_channel_create.cc:178] grpc_secure_channel_create(creds=0x7fd5bc850f70, target=secretmanager.googleapis.com:443, args=0x7fd583065c50, reserved=(nil))
init.cc:167]                grpc_init(void)
client_channel.cc:1099]     chand=0x7fd5bca91bb8: creating client_channel for channel stack 0x7fd5bca91b48
...
timer_manager.cc:188]       sleep for a 1001 milliseconds
...
timer_manager.cc:188]       sleep for a 1001 milliseconds
...

So, we a channel is created... and then nothing. No call, no operations. So the python code is sitting there waiting for gRPC to make this call, which it never does.



from Why does this gRPC call from the Google Secret Manager API hang when run by Apache?

Automated legend creation for 3D plot

I'm trying to update below function to report the clusters info via legend:

color_names = ["red", "blue", "yellow", "black", "pink", "purple", "orange"]

def plot_3d_transformed_data(df, title, colors="red"):
 
  ax = plt.figure(figsize=(12,10)).gca(projection='3d')
  #fig = plt.figure(figsize=(8, 8))
  #ax = fig.add_subplot(111, projection='3d')
  

  if type(colors) is np.ndarray:
    for cname, class_label in zip(color_names, np.unique(colors)):
      X_color = df[colors == class_label]
      ax.scatter(X_color[:, 0], X_color[:, 1], X_color[:, 2], marker="x", c=cname, label=f"Cluster {class_label}" if type(colors) is np.ndarray else None)
  else:
      ax.scatter(df.Type, df.Length, df.Freq, alpha=0.6, c=colors, marker="x", label=str(clusterSizes)  )

  ax.set_xlabel("PC1: Type")
  ax.set_ylabel("PC2: Length")
  ax.set_zlabel("PC3: Frequency")
  ax.set_title(title)
  
  if type(colors) is np.ndarray:
    #ax.legend()
    plt.gca().legend()
    
  
  plt.legend(bbox_to_anchor=(1.04,1), loc="upper left")
  plt.show()

So I call my function to visualize the clusters patterns by:

plot_3d_transformed_data(pdf_km_pred,
                         f'Clustering rare URL parameters for data of date: {DATE_FROM}  \nMethod: KMeans over PCA \nn_clusters={n_clusters} , Distance_Measure={DistanceMeasure}',
                         colors=pdf_km_pred.prediction_km)

print(clusterSizes)

Sadly I can't show the legend, and I have to print clusters members manually under the 3D plot. This is the output without legend with the following error: No handles with labels found to put in legend. img

I check this post, but I couldn't figure out what is the mistake in function to pass the cluster label list properly. I want to update the function so that I can demonstrate cluster labels via clusterSizes.index and their scale via clusterSizes.size

Expected output: As here suggests better using legend_elements() to determine a useful number of legend entries to be shown and return a tuple of handles and labels automatically.



from Automated legend creation for 3D plot

Why do I get 'MySQL server has gone away' after running a Telegram bot for some hours?

I'm building a Django (ver. 3.0.5) app that uses mysqlclient (ver. 2.0.3) as the DB backend. Additionally, I've written a Django command that runs a bot written using the python-telegram-bot API, so the mission of this bot is to run indefinitely, as it has to answer to commands anytime.

Problem is that approximately 24hrs. after running the bot (not necessarily being idle all the time), I get a django.db.utils.OperationalError: (2006, 'MySQL server has gone away') exception after running any command.

I'm absolutely sure the MySQL server has been running all the time and is still running at the time I get this exception. The MySQL server version is 5.7.35.

My assumption is that some MySQL threads get aged out and get closed, so after reusing them they won't get renewed.

Has anyone bumped into this situation and knows how to solve it?

Traceback (most recent call last):
  File "/opt/django/gip/venv/lib/python3.6/site-packages/telegram/ext/dispatcher.py", line 555, in process_update
    handler.handle_update(update, self, check, context)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/telegram/ext/handler.py", line 198, in handle_update
    return self.callback(update, context)
  File "/opt/django/gip/gip/hospital/gipcrbot.py", line 114, in ayuda
    perfil = get_permiso_efectivo(update.message.from_user.id)
  File "/opt/django/gip/gip/hospital/telegram/funciones.py", line 33, in get_permiso_efectivo
    u = Telegram.objects.get(idtelegram=userid)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/query.py", line 411, in get
    num = len(clone)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/query.py", line 258, in __len__
    self._fetch_all()
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/query.py", line 1261, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/query.py", line 57, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1151, in execute_sql
    cursor.execute(sql, params)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 74, in execute
    return self.cursor.execute(query, args)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/opt/django/gip/venv/lib/python3.6/site-packages/MySQLdb/connections.py", line 259, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (2006, 'MySQL server has gone away')

---- EDIT 1 ----

I already tried changing the Django settings.py file so I set an explicit value for CONN_MAX_AGE, and I also set a value for the MySQL client wait_timeout parameter, being CONN_MAX_AGE lower than wait_timeout.

settings.py:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/opt/django/gip/gip/gip/my.cnf',
        },
        'CONN_MAX_AGE': 3600,
    }
}

my.cnf:

[client]
...
wait_timeout = 28800

Unfortunately, the behavior is exactly the same: I get an exception approximately 24hrs. after running the bot.

---- EDIT 2 ----

Setting CONN_MAX_AGE to None won't make any difference either.



from Why do I get 'MySQL server has gone away' after running a Telegram bot for some hours?

AudioContext sound does not start immediately

Platform: Electron 13.1.7

Extra Library: Storyteller Engine https://github.com/hb432/storyteller

I created this XPlayer class (as part of storyteller) so I could play audio with the AudioContext API.

/** An audio player based on the AudioContext API. */
class XPlayer {
   /** The buffer this player should use as a source. */
   buffer: AudioBuffer;
   /**
    * Controls the speed and pitch of the audio being played. A value of `1` will result in no alterations to the
    * source audio, a value of `2` will half the speed and increase the pitch by an octive, etc.
    */
   rate: AudioParam;
   /** The audio router to use for this object. */
   router: XRouter;
   /** Controls the master volume of the audio being played. */
   volume: AudioParam;
   /**
    * This player's state. Contains the base context, base gain node, placeholder rate node, and all currently active
    * `AudioBufferSourceNode` instances that are currently active and/or awaiting closure.
    */
   state = (() => {
      const context = new AudioContext();
      return {
         context,
         gain: context.createGain(),
         rate: context.createGain().gain,
         sources: [] as AudioBufferSourceNode[]
      };
   })();
   constructor (
      {
         buffer,
         rate = 1,
         router = (context, source) => source.connect(context.destination),
         volume = 1
      }: XPlayerProperties = {}
   ) {
      this.buffer = buffer || this.state.context.createBuffer(1, 1, 8000);
      this.rate = this.state.rate;
      this.router = router;
      this.volume = this.state.gain.gain;
      this.state.rate.value = rate;
      this.state.gain.gain.value = volume;
      this.router(this.state.context, this.state.gain);
      addEventListener('beforeunload', () => {
         this.stop();
         this.state.context.close();
         // @ts-expect-error
         this.state.context = null;
      });
   }
   /** Returns the audio source most recently initialized by this player. */
   source (): AudioBufferSourceNode | void {
      return this.state.sources[this.state.sources.length - 1];
   }
   /**
    * Initializes a new audio source and starts the audio from the beginning. If `stop` is specified as true, the
    * `player.stop` method will be called before the new audio source is initialized.
    */
   start (stop?: boolean) {
      stop && this.stop();
      const source = Object.assign(this.state.context.createBufferSource(), { buffer: this.buffer });
      source.connect(this.state.gain);
      source.playbackRate.value = this.rate.value;
      this.rate = source.playbackRate;
      source.start();
      this.state.sources.push(source);
      return source;
   }
   /** Stops, de-activates, and flushes any currently active audio sources. */
   stop () {
      for (const source of this.state.sources.splice(0, this.state.sources.length)) {
         source.stop();
         source.disconnect(this.state.gain);
      }
   }
   /** Returns the current time of this player's associated audio context. */
   time () {
      return this.state.context.currentTime;
   }
}

However, I am having trouble actually starting the audio when the page is loaded. There seems to be some kind of "global delay" impacting the start time of all AudioContexts equally. I can queue up several XPlayer.start() calls, but no audio comes out until a specific moment, at which point every queued up bit of audio starts all at once. After that, audio works as intended.

Even stranger than that, the actual time it takes for the audio to start is ALWAYS the same across any reload -- precisely 30 seconds. No more, no less. It's always 30 seconds before the delayed audio plays.

I inspected the value of context.currentTime and it seems it remains stationary until the audio does its thing at 30 seconds. Something is preventing the timer from moving forward. There are no errors in console during all of this.



from AudioContext sound does not start immediately

Scrapy Splash, How to deal with onclick?

I'm trying to scrape the following site

I'm able to receive a response but i don't know how can i access the inner data of the below items in order to scrape it:

I noticed that accessing the items is actually handled by JavaScript and also the pagination.

What should i do in such case?

enter image description here

Below is my code:

import scrapy
from scrapy_splash import SplashRequest


class NmpaSpider(scrapy.Spider):
    name = 'nmpa'
    http_user = 'hidden' # as am using Cloud Splash
    allowed_domains = ['nmpa.gov.cn']

    def start_requests(self):
        yield SplashRequest('http://app1.nmpa.gov.cn/data_nmpa/face3/base.jsp?tableId=27&tableName=TABLE27&title=%E8%BF%9B%E5%8F%A3%E5%8C%BB%E7%96%97%E5%99%A8%E6%A2%B0%E4%BA%A7%E5%93%81%EF%BC%88%E6%B3%A8%E5%86%8C&bcId=152904442584853439006654836900', args={
            'wait': 5}
        )

    def parse(self, response):
        goal = response.xpath("//*[@id='content']//a/@href").getall()
        print(goal)


from Scrapy Splash, How to deal with onclick?

Sunday 29 August 2021

Text gets flushed to initial state after applying wrapping function in D3

I have following function for wrapping text in D3 taken from this answer:

function wrap(text, width) {
    text.each(function () {
        var text = d3.select(this),
            words = text.text().split(/\s+/).reverse(),
            word,
            line = [],
            lineNumber = 0,
            lineHeight = 1.1, // ems
            x = text.attr("x"),
            y = text.attr("y"),
            dy = 0, //parseFloat(text.attr("dy")),
            tspan = text.text(null)
                        .append("tspan")
                        .attr("x", x)
                        .attr("y", y)
                        .attr("dy", dy + "em");
        while (word = words.pop()) {
            line.push(word);
            tspan.text(line.join(" "));
            if (tspan.node().getComputedTextLength() > width) {
                line.pop();
                tspan.text(line.join(" "));
                line = [word];
                tspan = text.append("tspan")
                            .attr("x", x)
                            .attr("y", y)
                            .attr("dy", ++lineNumber * lineHeight + dy + "em")
                            .text(word);
            }
        }
    });
}

I initialize the nodes in a following manner:

const nodes = [1, 2, 3]
const nodesTextsEnter = vis.g.selectAll('text.node').data(nodes).enter()
const nodesTextsUpdate = vis.g.selectAll('text.node').data(nodes)
vis.g.selectAll('text.node').data(nodes).exit().remove()
        
nodesTextsEnter.append('text')
                .attr('x', (d, i) => i*30)
                .attr('y', (d, i) => i*30)
                .attr('text-anchor', 'middle')
                .attr('dominant-baseline', 'middle')
                .attr('font-size', 12)
                .style('cursor', 'pointer')
                .attr('class', 'node')
                .attr('opacity', '0')
                .transition()
                .attr('opacity', '1')
                .attr('x', (d, i) => i*30)
                .attr('y', (d, i) => i*30)
                .text('This is a long text')
                .call(wrap, 30)

nodesTextsUpdate
                .transition(transitionDuration)
                .attr('x', (d, i) => i*30)
                .attr('y', (d, i) => i*30)
                .text('This is a long text')
                .call(wrap, 30)

The problem is that while debugging wrap function, I see all the wrapping changes happening to the nodes, but then at some point (specifically, I figured it is happening on timerFlush() in wake function of timer.js) content of nodes gets flushed as though it was never wrapped.

Why this could be happening? Any ideas are welcome.



from Text gets flushed to initial state after applying wrapping function in D3