Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when a value in a CSV file contains quotes. #31

Open
Manasha11 opened this issue Jul 18, 2022 · 0 comments
Open

Error when a value in a CSV file contains quotes. #31

Manasha11 opened this issue Jul 18, 2022 · 0 comments

Comments

@Manasha11
Copy link
Contributor

Description:
The below errors are thrown from the CSV source mapper when processing a CSV file which includes quotes between the delimiters.

ERROR {io.siddhi.extension.map.csv.sourcemapper.CSVSourceMapper} - [Error] when converting the event from CSV message: 114a|0163320121|"D" FURNISHING POINT|175187294 to Siddhi Event in the stream FileDataStream of siddhi CSV input mapper. java.lang.RuntimeException: java.io.IOException: (line 1) invalid char between encapsulated token and delimiter
	at org.apache.commons.csv.CSVParser$1.getNextRecord(CSVParser.java:444)
	at org.apache.commons.csv.CSVParser$1.hasNext(CSVParser.java:454)
	at io.siddhi.extension.map.csv.sourcemapper.CSVSourceMapper.convertToEvents(CSVSourceMapper.java:290)
	at io.siddhi.extension.map.csv.sourcemapper.CSVSourceMapper.mapAndProcess(CSVSourceMapper.java:230)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:200)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:144)
	at io.siddhi.extension.io.file.processors.FileProcessor.receive(FileProcessor.java:168)
	at org.wso2.transport.file.connector.sender.VFSClientConnector.send(VFSClientConnector.java:367)
	at io.siddhi.extension.io.file.listeners.FileSystemListener.onMessage(FileSystemListener.java:197)
	at org.wso2.transport.remotefilesystem.server.RemoteFileSystemConsumer.consume(RemoteFileSystemConsumer.java:164)
	at org.wso2.transport.remotefilesystem.server.connector.contractimpl.RemoteFileSystemServerConnectorImpl.poll(RemoteFileSystemServerConnectorImpl.java:56)
	at io.siddhi.extension.io.file.FileSourcePoller.run(FileSourcePoller.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: (line 1) invalid char between encapsulated token and delimiter
	at org.apache.commons.csv.Lexer.parseEncapsulatedToken(Lexer.java:275)
	at org.apache.commons.csv.Lexer.nextToken(Lexer.java:152)
	at org.apache.commons.csv.CSVParser.nextRecord(CSVParser.java:500)
	at org.apache.commons.csv.CSVParser$1.getNextRecord(CSVParser.java:441)
	... 18 more

ERROR {io.siddhi.core.stream.input.source.SourceMapper} - Error while processing '114a|0163320121|"D" FURNISHING POINT|175187294', for the input Mapping 'csv' for the stream 'FileDataStream'. io.siddhi.core.exception.MappingFailedException
	at io.siddhi.extension.map.csv.sourcemapper.CSVSourceMapper.mapAndProcess(CSVSourceMapper.java:247)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:200)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:144)
	at io.siddhi.extension.io.file.processors.FileProcessor.receive(FileProcessor.java:168)
	at org.wso2.transport.file.connector.sender.VFSClientConnector.send(VFSClientConnector.java:367)
	at io.siddhi.extension.io.file.listeners.FileSystemListener.onMessage(FileSystemListener.java:197)
	at org.wso2.transport.remotefilesystem.server.RemoteFileSystemConsumer.consume(RemoteFileSystemConsumer.java:164)
	at org.wso2.transport.remotefilesystem.server.connector.contractimpl.RemoteFileSystemServerConnectorImpl.poll(RemoteFileSystemServerConnectorImpl.java:56)
	at io.siddhi.extension.io.file.FileSourcePoller.run(FileSourcePoller.java:46)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Affected Product Version:
SI 1.1.0

@Manasha11 Manasha11 changed the title Error when a value in a CSV file includes quotes. Error when a value in a CSV file contains quotes. Jul 18, 2022
Manasha11 added a commit to Manasha11/siddhi-map-csv that referenced this issue Jul 19, 2022
AnuGayan added a commit that referenced this issue Jul 26, 2022
lasanthaS added a commit that referenced this issue Jul 27, 2022
lasanthaS added a commit that referenced this issue Jul 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant