Tuesday, 18 June 2019

SQL on top of apache arrow in-browser?

I have data that is stored on a client's browser in-memory. For example, let's say the dataset is as follows:

"name" (string),     "age" (int32),   "isAdult" (bool)
"Tom"          ,     29               1
"Tom"          ,     14               0
"Dina"         ,     20               1

I would like to run non-trivial SQL statements on this data in javascript, such as:

SELECT name, GROUP_CONCAT(age ORDER BY age) ages
FROM arrowData a1 JOIN arrowData a2 USING (name)
WHERE a1.isAdult != a2.isAdult

And I would get:

"name" (string),      "ages" (string)
"Tom"                 "14,29"

The data that I have in javascript is stored in as apache Arrow (also used in connection with Perspective), and I'd like to execute SQL on that apache Arrow data as well. As a last resort, I think it would be possible to use sqllite in wasm, but I'm hoping there might be a simpler way where I can query the Arrow data directly, without having to move it all into a sqllite store in order to execute a query on it.

Are there any ways to do this?



from SQL on top of apache arrow in-browser?

No comments:

Post a Comment