Stop or abort a run remotely#

You can stop or abort an active run in the experiments table.

When you stop a run, it's considered to be finished. You can use this option, for example, to stop a monitoring script or to signal that the training is sufficient.
Aborting a run is similar to stopping it, but it will be marked as a failure (the state is set to failed). You can use this option to manually mark a run as unsuccessful.

To send a remote stop or abort request:

In the experiments table, click the icon next to the run ID.
Select Remote stop or Remote abort.

Neptune sends the signal to all processes that have an active connection to the run. The exact behavior depends on whether the psutil library is installed on the machine running the process. (By default, the psutil library is installed with the Neptune client.)

If psutil is installed, the current process and its children are aborted by sending SIGTERM. If the processes are not terminated after the grace period, SIGKILL is sent.
If psutil is not installed, we make a best effort to stop the process – for example, by invoking os.exit() – but it's not guaranteed to work in every environment.