From the course: Fine-Tune Your LLMs

Unlock the full course today

Join today to access over 23,000 courses taught by industry experts.

Solution: Prepare and upload data for fine-tuning

Solution: Prepare and upload data for fine-tuning

From the course: Fine-Tune Your LLMs

Start my 1-month free trial Buy for my team

Solution: Prepare and upload data for fine-tuning

“

(upbeat music) - [Instructor] How did you do? Let's look at how I solve this challenge. Here we are in code spaces. I've already executed all the cells in this notebook for efficiency. Don't forget to put your API key in a local environment, variable or hard code it within the notebook before executing this code. The first step is to convert the JSON file to the JSONL format, which is easy to do in Python. Let's scroll down to the function. Here we have the JSON to JSONL function. It writes a new output file in the proper format. There's also a helper function that confirms the file format, making sure that it's valid and another function called num tokens from messages that helps us count the tokens in order to estimate the cost. Here in cell four, I'm executing the JSON to JSON lines function. It's going to take as input, the customs support.json file, and it's going to produce this output.jsonl file and we can check over here and see that the file was actually created. The next…

Contents