Tuesday, 21 September 2021

How to make javascript regex match all lines when piping file

If I run my regex on the data as a string I have no issues my three lines get matched.

https://regex101.com/r/pHsTvV/1

const regex = /(?<email>((?:[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])))\s*\|\s*(?<name>([a-zA-Z]{2,}\s[a-zA-Z]{1,}'?-?[a-zA-Z]{2,}\s?([a-zA-Z]{1,})?))\s*\|\s*(?<address>.*)\s*\|\s*(?<country>(\w|\.|\s*){1,})\s*\|\s*(?<phone>(\d|-|\ |\+|\(|\)|\.|\/){7,})/gm;
const str = `john.doe@gmail.test| John Doe| 160 Boston Rd| Chelmsford MA 11824| United States| 00088782000
jane.doe@aol.test| Jane Doe| 8415 45th St| Lyons IL 60534| United States| 0005800000
alicia.random123@gmail.test| Alicia Random| BLK 8, City Point| No.58 Wing Shun Street| Tsuen Wan| Not in U.S.| +00092262000`;

const lines = str.split('\n')
lines.forEach(line => {
    const test = regex.exec(str)
    if (test && test.groups) {
        console.dir(test.groups)
    } else {
        console.log('could not match')
    }
});

However when I load my data from a txt file javascript always gives me one out of two lines not matched:

const regex = /(?<email>((?:[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])))\s*\|\s*(?<name>([a-zA-Z]{2,}\s[a-zA-Z]{1,}'?-?[a-zA-Z]{2,}\s?([a-zA-Z]{1,})?))\s*\|\s*(?<address>.*)\s*\|\s*(?<country>(\w|\.|\s*){1,})\s*\|\s*(?<phone>(\d|-|\ |\+|\(|\)|\.|\/){7,})/gm;
import * as fs from 'fs';
import * as path from 'path';
import * as es from 'event-stream';
const filePath = path.join(process.cwd(), 'data/test.txt')
var s = fs.createReadStream(filePath)
    .pipe(es.split())
    .pipe(es.mapSync(function (line: string) {
        let values = regex.exec(line.trim())
        if (values && values.groups) {
            console.dir(values.groups)
        } else {
            console.log(`COULD NOT MATCH`)
            console.log(line)
        }
    }).on('error', function (err) {
        console.log('Error while reading file.', err);
    })
        .on('end', function () {
            console.log('Read entire file.')
        })
    )

The test.txt file is as follows:

john.doe@gmail.test| John Doe| 160 Boston Rd| Chelmsford MA 11824| United States| 00088782000
jane.doe@aol.test| Jane Doe| 8415 45th St| Lyons IL 60534| United States| 0005800000
alicia.random123@gmail.test| Alicia Random| BLK 8, City Point| No.58 Wing Shun Street| Tsuen Wan| Not in U.S.| +00092262000

even on a file with 100 lines it is always one line out of two that do not get matched. When I read the file the jane.doe@aol.test does not get matched

I have tried the following to see if its line specific:

const regex = /(?<email>((?:[a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])))\s*\|\s*(?<name>([a-zA-Z]{2,}\s[a-zA-Z]{1,}'?-?[a-zA-Z]{2,}\s?([a-zA-Z]{1,})?))\s*\|\s*(?<address>.*)\s*\|\s*(?<country>(\w|\.|\s*){1,})\s*\|\s*(?<phone>(\d|-|\ |\+|\(|\)|\.|\/){7,})/gm;
const uniqueStr = `jane.doe@aol.test| Jane Doe| 8415 45th St| Lyons IL 60534| United States| 0005800000`

const test = regex.exec(uniqueStr)
if (test && test.groups) {
    console.dir(test.groups)
} else {
    console.log('could not match')
    console.log(uniqueStr)
}

This does not match but if I try the regex on regex101 there are no matches issue.

https://regex101.com/r/52kpRD/1



from How to make javascript regex match all lines when piping file

No comments:

Post a Comment