NextJS Dynamic List Not Updating on Delete/Update

Currently, my react/nextjs dynamic list is not updating correctly, I've given an array to map so it'll show a new row on frame update since it's stored on useState, Usually, the issue is with the key of the list that is not unique, but my key is unique

Heres my code

const [allSelectedMaterials, setAllSelectedMaterials] = useState([]) // This variable stores a javascript object into the array
{console.log('-- Updated --')}
{allSelectedMaterials.map((material, i) => {
    const key = Object.keys(material).toString()
    console.log(`${i} - [${key}] ${material}`)

    return (
        <div key={key}>
            <Row className='align-items-center'>
                <Col xs='auto'>
                    <h6>{material[key].name}</h6>
                </Col>
                <Col>
                    <Button variant='danger' className='mb-1' onClick={() => handleDeleteMaterial(key)}>
                        Remove
                    </Button>
                </Col>
            </Row>
            <InputGroup>
                <InputGroup.Text>
                    <Image src={material[key].image} className={`${styles.allMaterialsImage}`} />
                </InputGroup.Text>
                <Form.Control type='number' min={1} ref={(e) => (selectedMaterials.current[i] = e)} required />
            </InputGroup>
            <div className='mt-1' />
        </div>
    )
})}

The problem is after I added the first item on the list and adding a new one it won't update the view, here's the console output

And here's me adding a second entry to the list

It clearly says on the log that the array (stored in useState) is updated but it's not changing the view it's still the same as the previous one. But if I reupdate the frame by updating any useState variable it updated the list

Update: Here's my code for adding new material (loadedMaterials is just a list of materials that is retrieved from an API)

const handleAddSelectedMaterial = () => {
    loadedMaterials.map((material) => {
        const key = Object.keys(material)
        if (key == currentSelectedMaterial) {
            let _material
            if (allSelectedMaterials.length > 0) _material = allSelectedMaterials
            else _material = []

            _material.push({ [material[key].id]: material[key] })
            setAllSelectedMaterials(_material)
        }
    })
}

from NextJS Dynamic List Not Updating on Delete/Update