Cook's Country vs. the Robots: Who Can Make a Better Recipe?

So you can only imagine how I felt when the recent news about super-intelligent chatbots such as OpenAI’s ChatGPT started circulating.

Could it write an article about its very specific love for Australian Fairy Bread? Could it wax lyrical about the best technique to roll pie dough into a perfect circle? Even worse, could it develop a failproof recipe for a delicious chocolate chip cookie?

I had to find out. So, naturally, I put it to the test.

I asked ChatGPT to write me the very best chocolate chip cookie recipe it could come up with, and decided to bake it exactly as written to see how it measured up against our own rigorously tested recipe for the fan-favorite M&M Cookies (with the M&Ms subbed out for chocolate chips for an even playing field).

The ingredient list was certainly nothing to sniff at. It didn’t call for comically large portions of one thing, or an impossibly small amount of another. The chocolate-to-cookie ratio was decent. So far I was impressed and, honestly, a little nervous.

But it was once I got to actually making the recipe that I noticed some seemingly minor, but ultimately substantial, issues.

Having creamed together butter and sugar and combined my wet ingredients, I then noticed the chatbot’s recipe called for me to gradually whisk in the combined dry ingredients. Did you catch that? Whisk.

Knowing that I was honorbound to following the recipe exactly, I painstakingly added small amounts of the flour, baking soda, and salt mixture and whisked it into the creamy dough. This, reader, was no easy task.

Despite working with ¼ cup of dry ingredients at a time, the inside of my whisk quickly became caked with unmixed dough, leaving clumps of dry flour around the bowl. With each new addition the mixture became stiffer and the small spines of my whisk struggled against the dough, failing to incorporate the ingredients fully.

It took about 15 minutes of bicep-breaking work to fully mix the dry ingredients into the wet to result in what I deemed a workable dough. From there, it was into the oven and then onto a wire rack to fully cool.

The cookies came out perfectly fine—the edges were golden and crispy, the centers soft and cakey. They were decent chocolate chip cookies, and I would happily eat them again. But I staunchly refuse to ever be put through the backbreaking labor of making them again.

In comparing the AI’s cookies to our M&M cookies—a recipe that required hundreds of bake-throughs to ensure a perfect method and delicious results—there was frankly no competition: not in result nor in process. Why? Simple human trial and error.

Through all of our many tests, we increased the baking soda in the dough and upped the oven temperature to 425 degrees, pulling the cookies out sooner than most recipes would call for. Each of these strategies resulted in cookies that stayed soft and chewy—and fresher—for longer.

In fact, these cookies are so popular among our colleagues here at America's Test Kitchen that whenever a slightly tweaked recipe was being baked, lines would form out the door of the test kitchen to give the new batch a taste. They’re legendary around these halls.

Don’t believe me? Give them a go. Score a goal for the humans.