Use click features for project creation prompts

[lrcouto]

Description

Replaces Cookiecutter prompts with the click.prompt method, without affecting functionality in kedro new.
We are still using some methods that Cookiecutter imports from Rich, mainly for exceptions, so at the moment it'll still not allow us to uninstall rich without downgrading Cookiecutter.

Prompt appearance is now defined used the click.style method, so it can be changed if we so desire.
Example appearance:

Development notes

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

• Read the contributing guidelines
• Signed off each commit with a Developer Certificate of Origin (DCO)
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes
• Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

[lrcouto]

The current problem with this is that when the --starter flag is used, the prompts are supposed to be different. By using the click prompt feature, the default prompts are evaluated before we have the chance to check if a starter is being used or not. I will see if there's any way to get around this issue.

[astrojuanlu]

I confirm that the current branch works at least!

IIUC this now purely uses Click. Is there a plan to try to restore the colors later on, maybe using https://click.palletsprojects.com/en/stable/utils/#ansi-colors ? (not necessarily right now, as it would probably complicate fixing the remaining tests)

[lrcouto]

IIUC this now purely uses Click. Is there a plan to try to restore the colors later on, maybe using https://click.palletsprojects.com/en/stable/utils/#ansi-colors ? (not necessarily right now, as it would probably complicate fixing the remaining tests)

Definitely! I want to make sure all of the functionality is as we expect first, but it would be nice to have an option for colors (with the added bonus of not relying on Rich)

[ElenaKhaustova]

@lrcouto, could you please clarify if colours should work and how we can check everything works as expected?

[lrcouto]

@lrcouto, could you please clarify if colours should work and how we can check everything works as expected?

They do not work yet, I'm implementing a color scheme at the moment.

It should not affect automated tests, but the visual appearance will also be slightly different, as those prompts from cookiecutter were importing the Prompt class from Rich. I'll post some screenshots with options once I finish implementing it.

[lrcouto]

Could look somewhat like this, for example:

[merelcht]

I think this is a good start of removing some cookiecutter functionality, but like mentioned in the ticket:

• https://click.palletsprojects.com/en/8.1.x/options/#choice-options
• https://click.palletsprojects.com/en/8.1.x/options/#prompting

These could be used to get rid of/simplify the validation we do for inputs to kedro new options like --tools and --example

I think it would be good to try and simplify the validation/prompting logic by using click functionality.

[merelcht]

Do we still need to use our own custom validation or is it possible to use click for that as well? I saw that was one of the suggestions on the original ticket #3878

[merelcht]

Especially lower down in the code we have pretty complex parsing/validation methods e.g. _parse_tools_input.

[lrcouto]

I'm looking into this right now. From what I had seen I think the options that click has might be a bit unspecific for what we need, but there's an option to apply custom validation logic.

[merelcht]

Yes for the more complicated options it might not be possible, but for --example and --telemetry we only need to check yes/no/y/n. I think you can do something like:

@click.option(
    "--example",
    "-e",
    "example_pipeline",
    help=EXAMPLE_ARG_HELP,
    type=click.Choice(["yes", "no", "y", "n"], case_sensitive=False),
)

[lrcouto]

That does work for yes/no options. One minor issue though is that the error message is not customizable though. On top of that, the example pipeline option received through a config file would still need to have some validation since it doesn't go through the CLI options, so we'd still need the function.

This is what the error message from click.Choice looks like:

As opposed to the one we have right now, from _validate_input_with_regex_pattern:

For interactive prompts, that are happening inside a loop, there would need to be some sort of logic to pass the click.Choice value for the different prompts. Could be done through the prompts.yml file maybe.

[lrcouto]

For that specific validate we could use one of the click Exception options, though I feel like they were more designed to be used with CLI options and not with prompts. I also personally like having finer control over the appearance of the error messages.

(Also, could we maybe move this user_input parameter to be inside the _Prompt class and validate it as we set it? The function is called only once, I don't think it necessarily needs to be a function)

[lrcouto]

Another possibility is using the callback parameter in click for custom validation. For example, this:

def _validate_yes_no(ctx, param, value) -> None:
    if not re.match(r"(?i)^\s*(y|yes|n|no)\s*$", value, flags=re.X):
        raise click.BadParameter(f"'{value}' is an invalid value for {param}. It must contain only y, n, YES, or NO (case insensitive).")
    return value
    
    @click.option("--telemetry", "-tc", "telemetry_consent", help=TELEMETRY_ARG_HELP, callback=_validate_yes_no)

Results in this:

Does require writing the callback functions though. And the output is the click Exception output.

[merelcht]

I personally think the default click exception ("Invalid value for..") is fine. But maybe @astrojuanlu has a different view on that.

[astrojuanlu]

I agree the default click exception is fine, considering the tradeoffs

[merelcht]

Can you use Click choice options here https://click.palletsprojects.com/en/stable/options/#choice-options to simplify the prompting and validation logic for example project?

[lrcouto]

After some exploration of the project creation flow (colloquially referred to as "gazing into the abyss") with the objective of simplifying it in some way, shape or form, we can extract some insights:

• The main complication of the project creation flow is that we can get user input in three different ways: through the CLI, throught prompts, or through a config file. This makes input validation a little tricky, because as of now we're having to validate it in several different points of the new function, which feels confusing to read.
• On top of that, tools validation is particulary confusing because the input is different depending on the source. Sometimes it's numbers. Sometimes it's names. Sometimes it's a string. Sometimes it's a list. The data is converted from one format to the other multiple times throughout the project creation flow and it's easy to lose track of which is which.
• That said, changing the prompts themselves to using Click's instead of Cookiecutter's was doable without affecting functionality. The appearance is different, but the upside is that we have more control over it.
• ...which is one step in the direction of being able to run kedro new without having Rich installed, though more work need to be done on that. Changing our own prompts is not going to be enough, because we are using Cookiecutters git cloning function and it imports the rich.prompt library. Even though we are using the function with its no_input option, it still tries to import the lib and will break if it does not find it.

For now what could be done, besides the aforementioned replacement of the prompts, is replacing the --telemetry CLI input validation with the click.Choice option (which prevents an incorrect error message to be displayed if the input is bad, it was treating it as if it was the --example flag), and doing some cleaning up on the _Prompt class.