Parse error: Failed to construct DelphiFile (line 5883:0 missing FORWARD at 'var')

[Scharrels]

Prerequisites

• This bug is in SonarDelphi, not SonarQube or my Delphi code.
• This bug has not already been reported.

SonarDelphi version

1.7.0

SonarQube version

10.5.1

Issue description

The following error occurs on multiple files in our project. Luckily, one of the files is FastMM5, making it possible to report this bug without sharing our own code. The following file causes the error:

https://github.com/pleriche/FastMM5/blob/version_505/FastMM5.pas

This causes the sonar-delphi to crash with the following stacktrace:

14:44:25.820 ERROR Error while processing <projectpath>\fastmm\FastMM5.pas

au.com.integradev.delphi.file.DelphiFile$DelphiFileConstructionException: Failed to construct DelphiFile (line 5883:0 missing FORWARD at 'var')
	at au.com.integradev.delphi.file.DelphiFile.setupFile(DelphiFile.java:161)
	at au.com.integradev.delphi.file.DelphiFile.from(DelphiFile.java:136)
	at au.com.integradev.delphi.symbol.SymbolTableBuilder.process(SymbolTableBuilder.java:284)
	at au.com.integradev.delphi.symbol.SymbolTableBuilder.processImportsWithInlineRoutines(SymbolTableBuilder.java:371)
	at au.com.integradev.delphi.symbol.SymbolTableBuilder.process(SymbolTableBuilder.java:293)
	at au.com.integradev.delphi.symbol.SymbolTableBuilder.indexUnit(SymbolTableBuilder.java:396)
	at au.com.integradev.delphi.symbol.SymbolTableBuilder.build(SymbolTableBuilder.java:477)
	at au.com.integradev.delphi.DelphiSensor.executeOnFiles(DelphiSensor.java:119)
	at au.com.integradev.delphi.DelphiSensor.execute(DelphiSensor.java:83)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:88)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:61)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:79)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:61)
	at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:82)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:226)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:205)
	at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:204)
	at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:200)
	at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:173)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:226)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:205)
	at org.sonar.scanner.bootstrap.SpringScannerContainer.doAfterStart(SpringScannerContainer.java:351)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:226)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:205)
	at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:138)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:226)
	at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:205)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:71)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:65)
	at org.sonarsource.scanner.lib.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:41)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.lib.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:62)
	at jdk.proxy3/jdk.proxy3.$Proxy2.execute(Unknown Source)
	at org.sonarsource.scanner.lib.InProcessScannerEngineFacade.doAnalyze(InProcessScannerEngineFacade.java:39)
	at org.sonarsource.scanner.lib.ScannerEngineFacade.analyze(ScannerEngineFacade.java:61)
	at org.sonarsource.scanner.cli.Main.analyze(Main.java:77)
	at org.sonarsource.scanner.cli.Main.main(Main.java:63)
Caused by: au.com.integradev.delphi.antlr.DelphiParser$ParserException: line 5883:0 missing FORWARD at 'var'
	at au.com.integradev.delphi.antlr.DelphiParser.reportError(DelphiParser.java:387)
	at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:613)
	at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
	at au.com.integradev.delphi.antlr.DelphiParser.forwardDirectiveSection(DelphiParser.java:32532)
	at au.com.integradev.delphi.antlr.DelphiParser.forwardRoutineHeading(DelphiParser.java:16915)
	at au.com.integradev.delphi.antlr.DelphiParser.forwardRoutine(DelphiParser.java:16097)
	at au.com.integradev.delphi.antlr.DelphiParser.routineImplementation(DelphiParser.java:15863)
	at au.com.integradev.delphi.antlr.DelphiParser.declSection(DelphiParser.java:3480)
	at au.com.integradev.delphi.antlr.DelphiParser.unitImplementation(DelphiParser.java:2002)
	at au.com.integradev.delphi.antlr.DelphiParser.unit(DelphiParser.java:1618)
	at au.com.integradev.delphi.antlr.DelphiParser.file(DelphiParser.java:524)
	at au.com.integradev.delphi.file.DelphiFile.createAST(DelphiFile.java:196)
	at au.com.integradev.delphi.file.DelphiFile.setupFile(DelphiFile.java:149)
	... 38 common frames omitted
Caused by: org.antlr.runtime.MissingTokenException: null
	at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:612)
	... 49 common frames omitted

Steps to reproduce

The easiest way to reproduce this is to do a DelphiLint analysis on the following file:

https://github.com/pleriche/FastMM5/blob/version_505/FastMM5.pas

This shows the same error as sonar-delphi.

Minimal Delphi code exhibiting the issue

https://github.com/pleriche/FastMM5/blob/version_505/FastMM5.pas

[zaneduffield]

Hi @Scharrels, thanks for reporting this issue.
I was able to reproduce it after setting sonar.delphi.toolchain to DCC64.

I reduced the problem to this minimal example

procedure Foo;
{$if False}
asm
{$else}
begin
  var x := @Foo;
{$endif}
end;

What's happening here is the lexer is switching into ASM mode before the conditional directive processor has eliminated the asm branch, and the ASM label token generated make no sense to the grammar. It's an issue related to #261.

[Cirras]

As @zaneduffield noted, this would be fixed with #261.

However, I think it's a bad enough problem to warrant a medium-term hack.
It might be as simple as popping the asm mode in the parser when we encounter a begin token.