The Paperspace Job Runner is designed for users who want to execute code (such as training a deep neural network) on a cluster of GPUs easily and without thinking about the underlying infrastructure.
A job is consists of:
- a collection of files (code, resources, etc)
- a docker container (with code dependencies and packages pre-installed)
- a command to execute (i.e. python main.py or nvidia-smi)
01. Initialize a project directory
You can create a job by going in to any directory and typing paperspace project init which will initialize a namespace with the current directory's name.
02. Submit a job
You are now ready to run a job (even without any code!). You can run:
paperspace jobs create --container Test-Container --command "nvidia-smi"
Your job will get uploaded to our cluster of machines. Behind the scenes, we are zipping the current working directory, creating a Docker container, and running the command you provided.
Note: the zipped upload of your working directory is limited to 100MB
03. Check your progress
Jobs can output in two ways: First, they can produce log output. For example, you should see the output of `nvidia-smi` by running paperspace jobs logs --tail. You can also check the console GUI to view the log output.
The second way your job can create output is by adding any file to the `/artifacts` directory within the container. After your job has completed (during the "teardown" process) we upload these files to a secure location. You can see these files in the web console or download them directly through the CLI.