New multi-course structure for training materials #486
Conversation
First step toward evolving Hello Nextflow to be more modular. Just content moves and stub creation (not updated content yet except some rough chopping where bits are getting split up).
✅ Deploy Preview for nextflow-training ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
work in progress
Much more step by step demonstration of how channels work + overall logical progression should make more sense now
Mainly to make it more clear what we're learning from sections 3 and 4
This one is really completely trivial now (no more nf-core style directory nesting). Remaining TODOs are just copy pasting the finalized code.
A few TODOs remaining, mainly updating console outputs etc
|
|
||
| --- | ||
|
|
||
| ## 4. Supply a batch of multiple values via the `params` system |
There was a problem hiding this comment.
Would this not be a good place to stop, and have a subsequent (possibly quite short) 'hello operators', using flatten and splitCsv as exemplars?
There was a problem hiding this comment.
As discussed in slack, I'll add some more explicit text calling out the importance of operators in section 4 and won't split anything out for now.
|
|
||
| 1. Making data flow from one process to the next | ||
| 2. Collecting outputs from multiple process calls into a single process call | ||
| 3. Passing more than one input through a channel |
There was a problem hiding this comment.
We just did that in the previous module, right?
There was a problem hiding this comment.
Do you mean "Passing more than one input through a channel"? Here it's about having multiple 'tracks' in the input channel, eg the data but also some parameters etc., as a opposed to 'several pieces of the same kind of data'. In general this is a point of confusion for which we should try to develop a controlled vocabulary, because 'multiple inputs' ends up being ambiguous
There was a problem hiding this comment.
Yes, I actually realised what you meant further down. This probably needs to be something like 'passing collections of elements to a single task' or something similar.
There was a problem hiding this comment.
Are you using 'collections of elements' in the same way I'm using 'input tracks in a channel'?
| @@ -0,0 +1,755 @@ | |||
| # Part 3: Hello Plumbing | |||
There was a problem hiding this comment.
| # Part 3: Hello Plumbing | |
| # Part 3: Hello Workflows |
Maybe? We haven't really had a workflow so far, just a single step- this marks the point where we actually move beyond a single process into genuine worflows.
There was a problem hiding this comment.
I'm totally open to changing the name. FWIW 'Plumbing' speaks to me because it feels directly evocative of what this one is about, ie specifically how do you connect the pipes. I worry that 'Workflows' is maybe a bit too generic; I see what you're going for but is that going to be obvious to a naive user? They might go "isn't it all about workflows?" Perhaps we can poll for opinions/suggestions.
There was a problem hiding this comment.
This was kind of conscious (kind of). I was thinking the first sentence could be along the lines "You thought you'd been thinking about workflows already, but real workflows do these things..."
There was a problem hiding this comment.
That works for when people are actually going through it, but will it make sense to someone who just takes a look at the table of contents? I'd like the titles to be as informative as possible.
| } | ||
| ``` | ||
|
|
||
| This is what we can write based on what you've learned so far. |
There was a problem hiding this comment.
Is this really true? We already know about variable path inputs, and the syntax isn't different for multiple files.
There was a problem hiding this comment.
Well we don't know yet that the syntax isn't different. As a naive user, I'm like ok, I'm confident in all of this, but for this next bit I need to check if the syntax is the same or if i need to do something special.
|
|
||
| ### Takeaway | ||
|
|
||
| You know how to pass more than one input through a channel. |
There was a problem hiding this comment.
| You know how to pass more than one input through a channel. | |
| You know how to pass collected inputs through a channel. |
To dissambuguate from the earlier examples with multiple items.
There was a problem hiding this comment.
I find this more ambiguous though; this reads like it's about the collected greetings, but it's not; it's about that parameter that constitutes an extra 'track' in the input channel. This is exactly what I was talking about in my other comment just now; the ways we talk about multiple inputs is inconsistent and confusing.
There was a problem hiding this comment.
To me 'passing more than one input through a channel' covers both things- you're doing it in both cases. The difference here is that we're passing multiple inputs to a single task (a task being a single iteration of a process).
There was a problem hiding this comment.
But that's not what this specific line is about; that's not the takeaway. The takeaway from this specific section is that we now know that if the process expects path foo and val bar, i have to write the process call as myProcess(foo, bar). Before this, all I knew to do was myProcess(foo). It doesn't have anything to do with the fact that we're giving myProcess all the foos (that was the previous section).
There was a problem hiding this comment.
OK, apologies for the quick comment, I conflated issues while multitasking and I've re-read.
In fact, the point of this section (section 3) is that we've added a second input to the process, and filled that from a new parameter, right? So both of our versions are wrong. It's nothing to do with multiple inputs through 'a' channel (in fact there are now 2 channels, though one is implicit).
Maybe something like:
You know how to add more than one input channel to a process, and use both inputs from the workflow
Text improvements Co-authored-by: Jonathan Manning <[email protected]>
Co-authored-by: Jonathan Manning <[email protected]>
A few updates left but this is almost complete
After reconsidering the review comments, realized that it is ok and actually better to introduce variable inputs and params in Hello World. Getting them out of the way before Channels is actually helpful. The trick is to not get bogged down in details about value versus queue channels. The rest is a bit streamlined as a results. Also made Operators intro more explicit. Also got started on some cleanup of the code files
- Space after view {...}
- Config tweaks
- demo to test profile
- param json before profiles
- Also renamed Run Nextflow to Nextflow Run
remove example where we ask participants to executing a run that we know will fail.
…lo-nextflow' into gvda-streamline-hello-nextflow
…extflow-io/training into gvda-streamline-hello-nextflow
Managed to get map() in there after all! Thanks Rob for the feedback that got me there
Required some adaptations to config
First step toward evolving Hello Nextflow to be more modular. Just content moves and stub creation (not updated content yet except some rough chopping where bits are getting split up).