SQL DB plugin and AWS Athena

[jessicaparks]

Is it possible to use the SQL DB plugin to query data from Athena? If not, is this a feature you're considering adding in the future?

[pditommaso]

This plugin could support any DB having a JDBC interface. AFAIK Athena does not provide one.

Therefore it should require an Athena native API implementation, which would be very interesting to have tho not have a concrete plan in the short term

[pditommaso]

@abhi18av being a JBDC for Athena, I think the goal is the add the library in this project and make some tests to verify it works properly.

[abhi18av]

Ah gotcha - mixed the two projects here due to the overlap with AWS Athena!

[abhi18av]

@pditommaso regarding this - the jar is not published on Maven. The driver is contained in a Zip file with the JARs for JDBC-4.1 and JDBC-4.2 https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC-2.0.25.1002/SimbaAthenaJDBC-2.0.25.1002.zip

I have a couple questions

1. Does the license here https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC-2.0.25.1001/doc/LICENSE.txt look good, to be distributed along with this plugin?

2. As per https://www.baeldung.com/gradle-dependencies-local-jar, it seems it's possible to add the jar directly via Filesystem. Then for distribution, we add the JDBC 4.2 JAR to lib folder and then compile and distribute with the plugin?

[abhi18av]

I've added the draft work here nextflow-io/nextflow#2500 , eager to hear your thoughts.

[jorgeaguileraseqera]

related

nextflow-io/nextflow#2890