Setting up the Kaggle API on your Paperspace machine

What is Kaggle? It’s an incredible platform that lets you practice your machine learning skills via competitions. Although I’m just getting started, it’s highly regarded as one of the best ways to accelerate your skills in the machine learning community.

Where does Paperspace come into this? For those without their own heavy-duty hardware, Paperspace allows you to rent the computing power you need for machine learning. It’s the ideal way for students taking the fast.ai courses to set up a machine learning environment.

This guide shows you how to get the Kaggle API working on a Paperspace machine that’s been set up using the fast.ai template. All text in the guide that’s in italics is a command for you to run on your Paperspace machine terminal e.g. cd .kaggle

Prerequisites

For this to work, you’ll need to:

• Setup Paperspace using this awesome guide 
• Sign up for a Kaggle account
• Verify your Kaggle account: On your profile, click account then use the phone verification option

Kaggle API setup

Once the above steps are done, we can begin. You’ll need to:

1. Get the kaggle API package by running: pip install kaggle this will install kaggle to /home/paperspace/.local/bin/kaggle
2. Get your API credentials from kaggle:
   1. Go to the account section of your kaggle account, your web address will look like: https://www.kaggle.com/<username>/account when you’re on the right page.
   2. Click the Create New API token button which will activate the download of a file called kaggle.json (this file has your API credentials)
   3. Open the kaggle.json file you just downloaded in a text editor, select all the text and copy it
3. On your Paperspace machine, go to /home/paperspace by running cd ~
4. Create a .kaggle folder by running: mkdir .kaggle 
5. Move into the folder by running: cd .kaggle 
6. With the credentials you copied earlier (step 2, part 3 above), replacye “your_username” and “your_key” with your details and run the command on terminal: ‘{“username”:”your_username”,”key”:”your_key”}’ >> kaggle.json e.g. ‘{“username”:”ml_wizard123″,”key”:”ab43ask563aec”}’ >> kaggle.json
7. Set the kaggle environment variable by running: export KAGGLE_CONFIG_DIR=/home/paperspace/.kaggle/ 
8. You may get the following error “Your Kaggle API key is readable by other users on this system!”. To fix this and secure your credentials on your system, run: chmod 600 /home/paperspace/.kaggle/kaggle.json

The Kaggle API should now be setup. Take a short break e.g. grab a coffee ☕️, dance to celebrate.

Kaggle API examples

To download data from a specific competition e.g. Blue Book for Bulldozers, you will need to:

1. Go to the competition page
2. Click the “join competition” option
3. Accept the competition rules

You can now use the API commands to interact with the Kaggle platform. For example, if I wanted to work with the Blue Book for Bulldozers competition, I could:

• Search for the specific competition reference by running: kaggle competitions list -s bluebook, this gives the reference bluebook-for-bulldozers  

• Find the list of data files for the blue book competition by running: kaggle competitions files -c bluebook-for-bulldozers 

• Download the Train.zip file by running: kaggle competitions download bluebook-for-bulldozers -f Train.zip 

Further resources

Here are some links I used to make this guide:

• Environment variables
• Kaggle API
• Kaggle API docs

We have now set up our competition environment. This means we can now get the all-important practice in our machine learning journey Thanks for reading!