[help] How to forget/stop a dispatched job once launched

[swbioinf]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Hello,

Apologies, I have not been able to find a solution online.

I have a pipeline with two (file output) steps: A1=>A2, and B1=>B2 (Two samples, same functions)

Job A2 is currently at 'dispatched' status, but I'm fairly sure it has failed. I'd like to cancel/forget it and start again. I assume for some reason it has finished without generating an error, and this is assumed to be quietly running on a cluster somewhere?

• Last time this happened I use tar_destroy() to delete the whole targets store. But I'd like to keep A1.
• I gather tar_cancel() is not it, as that's just for cancelling a job from within
• I've tried killing the pid returned by tar_process(), but it doesn't exist. I'm 99% sure the job has stopped anyway - none of the running processes look like it.
• I can change the code for my function, which of course does set A2 to outdated. For now that's what I've done. But what if I want to keep B2 also?

For information, I'm launching tar_make() as an rstudio workbench job in the background (using rstudio workbench pro). Default tar_make (no use of crew e.t.c). Typically, the stdout of these jobs are and accessible via the workbench interface, but for whatever reason the history of completed jobs has disappeared.

For future reference - is there a way to stop or forget or force-ignore a 'dispatched' target so I can try again? Perhaps I"m looking at this from the wrong direction?

Thanks,
Sarah.

[wlandau]

I would recommend stopping and restarting tar_make() in that situation. Except for pre-specified debugging, pipelines are completely automated, encapsulated, and hidden from the user by design (to support reproducibility).

[swbioinf]

Thanks @wlandau , ok that makes sense from a workflow-y perspective.

When you say 'stopping and restarting tar_make()', am I right to think that means I'd have to find and kill the process that tar_make is running in?

Thanks,
Sarah.

[wlandau]

am I right to think that means I'd have to find and kill the process that tar_make is running in?

Under the default settings, CTRL-C in your local R session should work. If you're running tar_make(as_job = TRUE), you'll need to locate the RStudio job running the pipeline. If you run tar_make(callr_function = callr::r_bg), you'll need to locate the background process.