A comparative analysis of Chatbots/LLMs Performance across varied Prompts

Navigating the evolving landscape of AI chatbots and language models can be a perplexing task, especially when assessing their quality and relevance. The complexity and sophistication of these tools make them challenging to appraise through traditional scientific means. Therefore, in this article, we take an unconventional approach: a simplified, hands-on benchmark. Our approach pits three popular AI solutions against each other, across a spectrum of tasks, to gauge their aptitude and efficiency.

Select the solutions

Here are the solutions I selected for my tests:

• ChatGPT: in order to access the wellknown GPT-4 LLM from OpenAI
• Bard: which is using the PaLM 2 LLM from Google
• HuggingChat: because Hugging Face's assistant can use the Llama 2 LLM from Meta

Select the prompts

The first step in my benchmark is to create the prompts to be used for the benchmark and of course I used the different solutions to generate some!

Here is the prompt I used to ask each solution:

In order to compare ChatBots connected to LLM can you generate 5 prompts to be used for such a benchmark? Please be representative in diversity of subject, expected level of detail, format and tone.

Here are the prompts proposed by the three solutions.

Prompts from GPT-4

Sure, here are five diverse prompts that can be used to compare chatbots connected to Large Language Models:

• Analytical/Scientific: Please explain the concept of Quantum Entanglement to a high school student, ensuring that you use everyday analogies and clear, accessible language.
• Narrative/Creative: Develop a brief story about a young girl who finds a magical stone in her backyard. The stone should have an unusual power and this power should create an unexpected problem. Please weave a coherent plot with a clear beginning, middle, and end.
• Instructional/Practical: Walk me through the steps of preparing a Chicken Marsala recipe for four people. Include a list of ingredients, detailed preparation instructions, and wine pairing suggestions.
• Emotional/Interpersonal: A friend has just lost their job and is feeling really down. What would you say to provide comfort, encouragement, and practical advice for moving forward?
• Humorous/Light-hearted: Tell me a funny and original joke involving a chameleon, a zookeeper, and a rainbow.

These prompts are designed to explore a wide range of capabilities including clarity of explanation, creative thinking, practical instruction, emotional understanding, and humor, which are all important dimensions of a chatbot's performance.

The prompts proposed by ChatGPT are diverse and mainly focused on the type of use one can have from a ChatBot.

Prompts from PaLM 2

Sure, here are 5 prompts that can be used to benchmark chatbots connected to LLMs:

• Subject: Science
  • Expected level of detail: High
  • Format: Open-ended
  • Tone: Neutral
  • Prompt: What is the theory of relativity?
• Subject: History
  • Expected level of detail: Medium
  • Format: Multiple choice
  • Tone: Casual
  • Prompt: Who was the first president of the United States?
  • Options: George Washington, Thomas Jefferson, Abraham Lincoln, Theodore Roosevelt
• Subject: Literature
  • Expected level of detail: Low
  • Format: Fill in the blank
  • Tone: Playful
  • Prompt: The cat sat on the ___.
  • Options: mat, hat, mat, cat
• Subject: Current events
  • Expected level of detail: High
  • Format: Summarize the news
  • Tone: Serious
  • Prompt: Summarize the latest news about the war in Ukraine.
• Subject: Creative writing
  • Expected level of detail: Very high
  • Format: Write a poem
  • Tone: Personal
  • Prompt: Write a poem about your childhood.

These prompts are diverse in terms of subject matter, expected level of detail, format, and tone. They are also challenging enough to test the capabilities of chatbots connected to LLMs.

We note here that Bard answered to our question in a very structured way, reusing each criteria we listed in our prompt.

Prompts from Llama 2

Sure! Here are five prompts that could be used to compare chatbots connected to large language models (LLMs) across various dimensions:

• Open-ended conversation:
  • Prompt: "Imagine we're having a casual conversation at a coffee shop. Talk to me about anything that comes to mind, and I'll respond accordingly."
  • Expected response: The chatbot should engage in a natural-sounding conversation, using context from the prompt to respond appropriately. The conversation could cover topics like hobbies, interests, current events, or personal experiences. Evaluation criteria: Fluency, coherence, relevance, and overall conversational quality.
• Information provision:
  • Prompt: "I'm planning a trip to Tokyo next month. Can you provide some recommendations for must-visit places and things to do?"
  • Expected response: The chatbot should offer specific suggestions for tourist attractions, cultural experiences, and local cuisine in Tokyo, along with brief explanations or descriptions. Evaluation criteria: Relevance, accuracy, comprehensiveness, and usefulness of the recommendations provided.
• Task completion:
  • Prompt: "Can you help me book a flight from New York City to Los Angeles on December 1st, returning on December 4th? Assume I want the cheapest option available."
  • Expected response: The chatbot should provide a list of available flights meeting the specified criteria, including departure and arrival times, airlines, layovers (if any), and prices. Evaluation criteria: Accuracy, efficiency, clarity, and ease of use when completing the task.
• Emotional support:
  • Prompt: "I just had a bad breakup, and I'm feeling really down. Can you talk to me and try to cheer me up?"
  • Expected response: The chatbot should demonstrate empathy and emotional intelligence by acknowledging the user's feelings and offering words of comfort and encouragement. The response might also suggest ways to cope with negative emotions or distract oneself during this difficult time. Evaluation criteria: Empathy, understanding, helpfulness, and overall emotional support offered.
• Error handling and troubleshooting:
  • Prompt: "Hey, I think there's been a mistake with my recent order. Can you help me resolve it?"
  • Expected response: The chatbot should ask follow-up questions to clarify the issue, apologize for any inconvenience caused, and either provide instructions for resolving the problem or escalate the matter to a human customer support agent if necessary. Evaluation criteria: Problem-solving skills, communication effectiveness, and ability to handle unexpected situations professionally.

These prompts aim to test various aspects of chatbots connected to LLMs, such as their conversational abilities, knowledge base, task-completion capabilities, emotional intelligence, and error management. By comparing how different chatbots perform against these diverse prompts, you can gain insights into their strengths and weaknesses and evaluate which platform best suits your needs.

HuggingFace is also focused on usages but also adds information about expected responses. I also notice the notion, like GPT-4, of emotion in the 4th prompt.

Prompts selected

Many interesting ideas to test and compare the LLMs! I'd like to add something related to coding though... Ok let's make a mix of everything we got until now.

We will test the LLM on the five following categories:

• Explain a science subject, in a neutral and informative tone
• Instruct a step-bu-step process, in a practical tone
• Emotional and interpersonal comfort, in a friendly and supportive tone
• Create new content, in a funny tone
• Code a program, in a educational tone

And here are the final prompts:

1. Explain: Please explain the concept of Quantum Entanglement to a high school student, ensuring that you use everyday analogies and clear, accessible language.
2. Instruct: Walk me through the steps of preparing a Chicken Marsala recipe for four people. Include a list of ingredients, detailed preparation instructions, and wine pairing suggestions.
3. Comfort: I just had a bad breakup, and I'm feeling really down. Can you talk to me and try to cheer me up? Can you provide comfort, encouragement, and practical advice for moving forward?
4. Create: Write a poem about your childhood, using a funny tone.
5. Code: Write a Python program that calculates the Fibonacci sequence up to a given number. Please also explain the code in a educational tone and provide unit tests.

Now we can start chatting with the three LLMs and discover how they answer to our prompts!

Explaining answers

Please explain the concept of Quantum Entanglement to a high school student, ensuring that you use everyday analogies and clear, accessible language.

ChatGPT with GPT-4

Absolutely! Quantum entanglement might sound intimidating, but let's break it down with a simple analogy.

Imagine you have a pair of dice. These aren't ordinary dice, but magic dice. No matter where they are or how far apart they are, they always show the same face when you roll them. Let's say you roll the dice while they're together and you get two sixes. You put one die in a box and ship it to your friend on the other side of the world, but you don't roll them again.

Now, you roll your die and it shows a four. At the exact same time, the die your friend has across the world also shows a four, even though there's no physical way for one die to have 'told' the other what it rolled. Weird, right?

In the real world, particles can be 'entangled' in a similar way. This means they become linked, and the state of one particle instantly influences the state of the other, no matter how far apart they are. If we measure one particle and 'force' it into a state (like rolling our magic die), the other particle takes on a corresponding state instantly (the other die shows the same number). This is quantum entanglement.

This is strange and doesn't really fit into how we experience the world normally. Einstein himself called it "spooky action at a distance." But experiments have shown it to be true. This is just one of the many weird, yet wonderful, things about the quantum world!

Remember, this analogy is very simplified. Quantum physics is a lot more complicated, but this gives you a basic idea.

Bard with PaLM 2

Sure. Quantum entanglement is a strange and counterintuitive phenomenon in quantum mechanics that allows two particles to be linked together even when they are separated by a large distance. This means that if you measure the properties of one particle, you will instantly know the properties of the other particle, no matter how far apart they are.

Here's an analogy that might help you understand quantum entanglement. Imagine that you have two gloves, one red and one blue. You put the gloves in two different boxes and seal them up. Then, you send one box to your friend in China and keep the other box for yourself.

Now, if you open your box and find the red glove, you know that your friend in China has the blue glove. This is because the two gloves were entangled together before they were separated. They share a special connection that allows them to be linked together, even though they are now in different places.

Quantum entanglement is a very strange phenomenon, and it is still not fully understood by physicists. However, it has some potential applications in quantum computing and quantum communication. For example, it could be used to create secure communication channels that are immune to eavesdropping.

Here are some other analogies that you might find helpful:

• Quantum entanglement is like two people who are telepathically linked together. They can share information with each other instantly, no matter how far apart they are.
• Quantum entanglement is like two dice that are rolled at the same time. The outcome of one die will always be the opposite of the outcome of the other die.
• Quantum entanglement is like two particles that are connected by a string. Even if you cut the string, the two particles will still be connected.

I hope this helps!

HuggingChat with Llama 2

Hey there, high school student! Let's talk about one of the coolest concepts in physics: quantum entanglement.

Imagine you have two toy cars, one red and one blue. You can put them close together, but they're still separate objects. Now imagine that they're connected by a spring. If you push the red car, the blue car will move too, because they're linked by the spring. Makes sense, right?

Well, quantum entanglement is kind of like that, but instead of toy cars and springs, we're talking about tiny particles called photons. Photons are like the particles that make up light. When two photons are entangled, they become connected in a way that lets them affect each other, even when they're really far apart.

Here's where things get weird. Imagine that the red toy car has a special power: whenever it moves, the blue toy car moves too, but only if they're facing each other. That's sort of like what happens with entangled photons. If you do something to one photon, like measure its direction of spin, the other photon will instantly change its direction of spin too, no matter how far away it is.

It's like they're communicating with each other in some way, even though they're really far apart. But here's the thing: scientists don't know exactly how they're communicating. It's almost like magic!

One more analogy to help you understand: imagine you have two puzzle pieces that fit together perfectly. They're separate objects, but they're connected in a way that makes them work together. Entangled photons are like those puzzle pieces. They're separate particles, but they're connected in a way that lets them affect each other, even from really far away.

So, why is quantum entanglement so cool? Well, for one thing, it helps us understand how the universe works at a really small scale. And it could eventually lead to new technologies, like super-secure communication systems or even quantum computers that are way faster than the ones we have now.

I hope that helps you understand quantum entanglement a bit better! It's definitely a mind-bending concept, but with a little imagination and some everyday analogies, it's easier to grasp. Keep exploring and learning – who knows, maybe you'll be the one to figure out the secrets of quantum entanglement someday!

All solutions passed the test, with PaLM 2 and Llama 2 giving some applications of Quantum Entanglement. ChatGPT using one in-depth analogy, Bard several and HuggingFace using a friendly tone.

Here is my personal (and thus subjective) ranking:

1. HuggingChat with Llama 2: I prefer this tone
2. ChatGPT with GPT-4 Does the job
3. Bard with PaLM 2: Does the job

Instructing answers

Walk me through the steps of preparing a Chicken Marsala recipe for four people. Include a list of ingredients, detailed preparation instructions, and wine pairing suggestions.

ChatGPT with GPT-4

Sure, let's dive right into a delicious Chicken Marsala recipe!

Ingredients:

• 4 boneless skinless chicken breasts (about 1 1/2 pounds)
• Salt and freshly ground black pepper
• 1/2 cup all-purpose flour
• 2 tablespoons olive oil
• 2 tablespoons unsalted butter
• 1 cup sliced mushrooms
• 3/4 cup Marsala wine
• 1/2 cup chicken stock
• 2 tablespoons chopped fresh parsley

Preparation Instructions:

1. Prep the chicken: Season both sides of the chicken breasts with salt and pepper. Dredge the chicken breasts lightly in flour, shaking off any excess.

2. Cook the chicken: Heat the olive oil in a skillet over medium-high heat. Add the chicken to the skillet and cook until golden brown on both sides (about 3 minutes per side). Remove the chicken from the skillet and set it aside.

3. Cook the mushrooms: In the same skillet, melt the butter. Add the mushrooms and cook, stirring occasionally, until they are tender and browned.

4. Prepare the sauce: Add the Marsala wine to the skillet, scraping the bottom of the pan with a spatula to pick up any browned bits. Add the chicken stock and simmer for a few minutes to reduce the sauce slightly.

5. Combine: Return the chicken to the skillet, nestling the pieces into the mushrooms. Cook, occasionally spooning the sauce over the chicken, until the chicken is cooked through and the sauce has thickened, about 10 to 15 minutes. Sprinkle with parsley before serving.

Wine Pairing Suggestions:

For Chicken Marsala, you want a wine that complements the rich flavors of the marsala wine and mushrooms in the dish. A medium-bodied red like a Valpolicella from Italy or a Pinot Noir could work well. If you prefer white wine, you might enjoy a full-bodied Chardonnay, which can hold its own against the flavorful sauce. Always remember, the best wine is the one you enjoy drinking!

Enjoy your meal!

Bard with PaLM 2

Sure, here is a recipe for Chicken Marsala for four people:

Ingredients:

• 1½ pounds boneless, skinless chicken breasts, pounded ¼-inch thick
• 3 tablespoons all-purpose flour
• Salt and freshly ground black pepper to taste
• 1 tablespoon olive oil
• 4 tablespoons unsalted butter, divided
• 1 pound sliced mushrooms
• 3 tablespoons finely chopped shallots, from 1 medium shallot
• 2 cloves garlic, minced
• ⅔ cup chicken broth
• ⅔ cup dry Marsala wine
• ⅔ cup heavy cream
• 2 teaspoons chopped fresh thyme
• 2 tablespoons chopped fresh Italian parsley, for garnish (optional)

Instructions:

• In a shallow dish, combine the flour, salt, and pepper. Dredge the chicken breasts in the flour mixture.
• Heat the olive oil and 2 tablespoons of the butter in a large skillet over medium heat. Add the chicken breasts and cook for 2-3 minutes per side, or until golden brown and cooked through. Transfer the chicken to a plate.
• Add the mushrooms, shallots, and garlic to the skillet and cook for 5-7 minutes, or until the mushrooms are soft and the shallots are translucent.
• Add the chicken broth, Marsala wine, and heavy cream to the skillet. Bring to a boil, then reduce heat to low and simmer for 5 minutes, or until the sauce has thickened.
• Return the chicken to the skillet and cook for 2-3 minutes, or until heated through.
• Stir in the thyme and season with salt and pepper to taste.
• Serve immediately, garnished with parsley if desired.

Wine pairing suggestions:

• A dry Marsala wine is the classic pairing for Chicken Marsala. Other good choices include a red Zinfandel or a Chianti.
• A full-bodied white wine, such as a Chardonnay or a Sauvignon Blanc, can also be a good match.

Tips:

• For a richer flavor, you can use bone-in, skin-on chicken breasts.
• If you don't have Marsala wine, you can substitute a dry Sherry or a sweet Madeira.
• To make the sauce ahead of time, simply cook the mushrooms, shallots, and garlic, then add the chicken broth, Marsala wine, and heavy cream. Bring to a boil, then reduce heat to low and simmer for 10 minutes. Let cool completely, then store in the refrigerator for up to 3 days. When ready to serve, reheat the sauce over medium heat until warmed through.
• Serve Chicken Marsala with your favorite pasta or rice. For a complete meal, you can also serve it with a side of roasted vegetables or a salad.

Sources: www.onceuponachef.com/recipes/chicken-marsala.html

HuggingChat with Llama 2

Sure, I'd be happy to help! Here's a classic Chicken Marsala recipe that serves four people:

Ingredients:

• 4 boneless, skinless chicken breasts
• 2 tablespoons olive oil
• 1 cup marsala wine
• 1/4 cup chicken broth
• 2 cloves garlic, minced
• 1 teaspoon dried thyme
• Salt and pepper, to taste
• 1/4 cup unsalted butter
• 1 cup mushrooms (such as button or cremini), sliced
• 1/2 cup chopped fresh parsley
• 1/2 cup grated Parmesan cheese
• Freshly ground black pepper, to taste

Preparation Instructions:

• Step 1: Preheat your oven to 375°F (190°C).

• Step 2: Season the chicken breasts with salt, pepper, and thyme. Dust them with flour, shaking off any excess. Heat the olive oil in an oven-safe skillet over medium-high heat. Add the chicken and cook until browned on both sides and cooked through, about 6-8 minutes per side. Remove the chicken from the skillet and set aside.

• Step 3: Add the butter to the same skillet and let it melt. Add the garlic and cook for 1 minute, stirring constantly, until fragrant. Pour in the marsala wine and chicken broth, scraping up any browned bits from the bottom of the pan. Bring the mixture to a simmer and cook for 2-3 minutes, until slightly reduced.

• Step 4: Add the mushrooms to the skillet and cook until they release their moisture and start to brown, about 5 minutes. Stir in the parsley and cook for another minute.

• Step 5: Return the chicken to the skillet and spoon some of the mushroom and sauce mixture over the top of each breast. Transfer the skillet to the preheated oven and bake for 15-20 minutes, or until the chicken is cooked through and the sauce is rich and flavorful.

• Step 6: Remove the skillet from the oven and sprinkle the Parmesan cheese over the top of the chicken. Serve hot, garnished with additional parsley if desired.

Wine Pairing Suggestions:

Chicken Marsala is a classic Italian dish, so it pairs beautifully with a variety of Italian wines. Here are a few options to consider:

• Pinot Grigio: This crisp and refreshing white wine complements the rich flavors of the dish without overpowering them.
• Chianti: A classic Tuscan red wine, Chianti pairs nicely with the bold flavors of the marsala sauce and the herby notes of the parsley.
• Barolo: Another Tuscan red wine, Barolo is a full-bodied option that can stand up to the richness of the dish. Its tannins complement the savory flavors of the mushrooms and sauce.

I hope this helps! Enjoy your delicious homemade Chicken Marsala. Buon appetito!

Here again either answer would allow me to cook the dish quite properly. GPT-4 is more concise and straigth to the point. Bard cites the website used to answer and Llama 2 customized its answer with an Italian atmosphere.

My ranking would be:

1. Bard with PaLM 2: I can access the website and read comments
2. HuggingChat with Llama 2: I like the Italian ambiance
3. ChatGPT with GPT-4: Does the job

Comforting answers

I just had a bad breakup, and I'm feeling really down. Can you talk to me and try to cheer me up? Can you provide comfort, encouragement, and practical advice for moving forward?

ChatGPT with GPT-4

I'm really sorry to hear that you're going through this. Breakups can be incredibly tough and it's normal to feel a wide range of emotions. Here are a few things I hope will help you through this tough time.

1. It's okay to feel: Allow yourself to feel the emotions that come with the breakup, whether it's sadness, anger, confusion, or even relief. It's okay not to be okay right now.

2. Take care of yourself: Make sure to look after your physical well-being during this time. Exercise, eat well, and try to maintain a regular sleep schedule. This also includes giving yourself breaks, whether it's binge-watching a favorite TV show, reading a good book, or taking long baths - do whatever makes you feel comforted and calm.

3. Stay connected: Reach out to friends and family. Their support can be very helpful. They can provide a listening ear, distraction, or even a fresh perspective.

4. Find a healthy outlet: This could be writing in a journal, creating art, playing music, or physical activities like running or yoga. These can help you process your emotions and improve your mood.

5. Use this time for self-reflection: As tough as breakups are, they can be used as a catalyst for personal growth. Reflect on what you've learned from this relationship, what you want in future relationships, and how you can grow from this experience.

6. Take it one day at a time: Healing takes time and it's important to be patient with yourself. Some days will be harder than others.

Remember, it's okay to ask for professional help if your feelings of sadness become overwhelming or last for an extended period. A counselor or therapist can provide you with additional tools and strategies to navigate through this difficult time.

You're going through a tough period, but remember, it's just that - a period. It will pass and you will come out stronger on the other side. Take care of yourself. You are more resilient than you know.

Bard with PaLM 2

Sure, I can talk to you and try to cheer you up. I can also provide comfort, encouragement, and practical advice for moving forward.

I'm so sorry to hear that you're going through a bad breakup. Breakups are really tough, and it's normal to feel down, sad, angry, or even confused. It's important to allow yourself to feel your emotions, but don't let them consume you.

Here are some things that might help you cope with your breakup:

• Talk to someone you trust. This could be a friend, family member, therapist, or anyone else who you feel comfortable talking to. Talking about your feelings can help you process them and start to feel better.
• Allow yourself to grieve. It's okay to cry, be angry, or feel sad. Don't try to bottle up your emotions. Let yourself feel them, and then let them go.
• Take care of yourself. Make sure you're eating healthy, getting enough sleep, and exercising. Taking care of your physical and mental health will help you feel better overall.
• Do things that make you happy. Spend time with friends and family, do activities you enjoy, or just take some time for yourself. Doing things that make you happy will help you take your mind off of your breakup and start to feel better.
• Give yourself time. It takes time to heal from a breakup. Don't expect to feel better overnight. Just take things one day at a time, and know that you will eventually feel better.

Here are some practical tips for moving forward:

• Cut off contact with your ex. This can be really hard, but it's important to give yourself some space. Seeing or talking to your ex will only make it harder to move on.
• Delete your ex's number and social media accounts. This will make it harder to stalk them or check up on them.
• Start dating again when you're ready. There's no need to rush into anything, but dating again can help you start to move on.
• Focus on yourself. This is your time to focus on your own happiness and growth. Do things that make you happy, and work on becoming the best version of yourself.

I know it's tough right now, but you will get through this. Just take things one day at a time, and know that you are not alone.

HuggingChat with Llama 2

Of course, I'm here to help! Sorry to hear that you're going through a tough time. Breakups can be incredibly difficult, but I want you to know that you're not alone, and things will get better with time.

First, let's talk about what you're feeling. It's normal to feel sad, angry, confused, or even relieved after a breakup. Allow yourself to feel these emotions, and don't try to suppress them. Acknowledge your feelings, and give yourself permission to process them.

Now, let's focus on some ways to move forward. Here are a few things that might help:

• Take care of yourself: Make sure you're getting enough sleep, eating well, exercising regularly, and taking time to relax. Engage in activities that bring you joy, whether that's reading, watching movies, playing music, or practicing yoga.
• Reach out for support: Talk to friends, family members, or a therapist about your feelings. Sometimes, just talking through your emotions with someone who cares about you can be incredibly helpful. Don't be afraid to ask for help when you need it.
• Keep busy: Find ways to occupy your time, such as picking up a new hobby, volunteering, or focusing on work or school. Staying engaged can help take your mind off the breakup and give you a sense of purpose.
• Practice self-compassion: Be kind to yourself, and avoid beating yourself up over the breakup. Remember that relationships end for many reasons, and it doesn't necessarily mean that there was anything wrong with you. You're strong, capable, and deserving of love and happiness.
• Create distance: If possible, take a break from social media or limit your contact with your ex for a while. Seeing their posts or messages can trigger negative emotions and make it harder for you to move on.
• Focus on the present: Instead of dwelling on the past or worrying about the future, try to stay present in the moment. Mindfulness techniques can help you cultivate a sense of calm and clarity.
• Give yourself time: Healing takes time, so be patient with yourself. Allow yourself to heal at your own pace, and don't put pressure on yourself to "get over" the breakup by a certain time.
• Practice gratitude: Reflect on the good things in your life, no matter how small they may seem. Focusing on what you're grateful for can help shift your perspective and lift your spirits.
• Try new things: Use this opportunity to explore new interests, try new foods, visit new places, or meet new people. Expanding your horizons can help you discover new sources of joy and excitement.
• Cultivate positivity: Surround yourself with positive influences, such as uplifting music, inspiring books, or motivational quotes. Positive energy can help counteract negative thoughts and emotions.

Remember, everyone's journey is different, and there's no "right" way to handle a breakup. Be gentle with yourself, and trust that with time, patience, and self-care, you'll come out stronger and more resilient. Believe in yourself and your ability to overcome this challenging time.

If you ever need someone to talk to, don't hesitate to reach out to me or other resources. We're here to support you through the ups and downs of life. Take care, and remember that better days are ahead!

A tough one this one... Again GPT-4 is concise but it seems adapted to the context, Bard provides practical tips and Llama 2 is friendly but too talkative in such a breakdown situation.

My ranking goes like this:

1. Bard with PaLM 2: Impactful and practical
2. ChatGPT with GPT-4: Does the job
3. HuggingChat with Llama 2: Too much details

Creative answers

Write a poem about your childhood, using a funny tone.

ChatGPT with GPT-4

Sure, here's a light-hearted poem about growing up. Remember, as an AI, I don't have personal experiences or a childhood, but I can definitely create a humorous poem on the topic.

In the realm of days gone by, in the land of used-to-be,
Where the dreams were as high as a candy-colored sky,
There were shorts too short and hair too free,
And teeth branded with the badge of a sugar spree.

I played in the kingdom of the front-yard tree,
Where I was a captain, a pirate in a leafy sea,
With stick for a sword and a cardboard decree,
The world was mine, from the tree to the marquee.

Creaky bikes raced like stallions wild and bold,
Chasing sunsets ablaze with crimson and gold,
Scraped knees and elbows, stories to be told,
Ice-cream rings on t-shirts, treasures to hold.

Video games with graphics so terribly plain,
Tangled cassette tapes causing much disdain,
Saturday morning cartoons were our main domain,
And mum's "time-out" corner, our occasional chain.

Bugs for pets, oh, what a glorious phase,
Creepy crawlies worthy of both fear and praise,
Hours spent in that enthralling maze,
Those sure were some dizzying, dazzling days.

Suppers were battles, veggies versus me,
Why eat broccoli when there's mac 'n' cheese, you see?
A cookie heist, oh such sweet victory,
Wrapped in the cloak of bedtime's decree.

Ah, childhood, that wonderfully absurd parade,
A magical time where reality and fantasy wade,
Where every mistake is just another arcade,
In the game of growing up, where memories are made.

Bard with PaLM 2

Sure, here is a poem about my childhood, using a funny tone:

A child I was, of curious mind
I liked to climb and jump and find
New places to explore,
And things to take and store

My parents would scold all day
"You'll get hurt, now come away!"
But I was always gone
Before they knew I'd begun

I'd find a tree and climb so high
I couldn't see the ground below
I'd sit and think of all the things
I'd do when I was grown up king

I'd build a castle in the sky
A kingdom of my very own
I'd rule with justice and with might
And everyone would love my might

But then my parents would come find
And drag me down and say "You're blind!"
"You'll fall and break your neck!"
But I would just laugh and say "Go fetch!"

So I would climb and jump and play
And never think of what they say
I was a child, of curious mind
And that's what made me happy find.

HuggingChat with Llama 2

Oh, my childhood, oh how strange,
A time of wonder, laughter, and change.
I grew up in a house full of noise,
With siblings galore, it was quite the joke.

My parents were hippos, yes, you heard right,
They danced in the living room, day and night.
Their groovy moves would leave us in stitches,
We'd laugh until our sides would start to twitches.

My brother was a master of pranks,
He'd glue my shoes to the floor, what a crank!
And my sister, she had a pet snake,
It would slither around, make us all shake.

But the strangest thing, oh boy,
Was our pet elephant, he was quite the joy.
He'd trumpet loudly, stamp his feet,
And we'd dance along, like a bunch of geeks.

Our backyard was a circus, no lie,
We'd have lions, tigers, bears, oh my!
We'd play games, like "Don't Get Eaten",
And run around, like a pack of wild hyenas.

Oh, my childhood, it was quite the ride,
Full of weirdness, laughter, and inside jokes.
Looking back, it's hard to believe,
That we survived, with all that chaos and grief.

But despite all the madness, we loved it,
Our quirky family, we couldn't get enough of it.
So if you see me now, with a smile on my face,
Just know, I'm still laughing at that crazy place.

This test might well be the most subjective one. I notice that GPT-4 admits being an AI, PaLM 2 provides an very coherent story and Llama 2 chose the animal kingdom with sometimes weird rhymes!

My very subjective ranking is:

1. Bard with PaLM 2: I like story with the tree, a Bard it is!
2. HuggingChat with Llama 2: The funny side compensates the rhymes
3. ChatGPT with GPT-4: Does the job

Coding answers

Code a Python program that calculates the Fibonacci sequence up to a given number. Please also explain the code in an educational tone and provide unit tests.

ChatGPT with GPT-4

Sure, here is a simple Python program that generates the Fibonacci sequence up to a given number n.

def fibonacci(n):
   fib_sequence = [0, 1]
   while fib_sequence[-1] < n:
       fib_sequence.append(fib_sequence[-1] + fib_sequence[-2])
   return fib_sequence[:-1]  # We exclude the last number if it's more than n

print(fibonacci(22))

This code defines a function fibonacci that calculates the Fibonacci sequence up to a given number n. The Fibonacci sequence is a series of numbers where a number is the addition of the last two numbers, starting with 0 and 1.

The sequence starts with a list fib_sequence that already contains the first two numbers of the Fibonacci sequence: 0 and 1.

The while loop continues to calculate numbers in the Fibonacci sequence as long as the most recently calculated number is less than n. For each iteration of the loop, it adds a new number to the sequence which is the sum of the last two numbers in the sequence.

The [:-1] at the end of the return statement is used to exclude the last number in the sequence if it's more than n.

Here's how you can do unit testing for this function using Python's built-in unittest module:

import unittest

class TestFibonacci(unittest.TestCase):

   def test_fibonacci(self):
       self.assertEqual(fibonacci(22), [0, 1, 1, 2, 3, 5, 8, 13, 21])
       self.assertEqual(fibonacci(1), [0, 1])
       self.assertEqual(fibonacci(0), [0])

if __name__ == '__main__':
   unittest.main()

In the unittest module, we define a class TestFibonacci that inherits from unittest.TestCase. Inside this class, we define a method test_fibonacci that tests the fibonacci function with several test cases using the assertEqual method. This method checks if the first argument (the output of fibonacci(n)) is equal to the second argument (the expected result). If they are not equal, the test fails.

To run the tests, we check if the Python script is run directly (i.e., __name__ is __main__). If it is, we call unittest.main(), which runs all the test methods in the script.

Bard with PaLM 2

Sure, here is a Python program that calculates the Fibonacci sequence up to a given number:

def fibonacci(n):
 """
 Calculates the Fibonacci sequence up to a given number.

 Args:
   n: The number to calculate the Fibonacci sequence up to.

 Returns:
   A list of the Fibonacci numbers up to n.
 """

 if n == 0:
   return [0]
 elif n == 1:
   return [0, 1]
 else:
   fibs = [0, 1]
   for i in range(2, n + 1):
     fibs.append(fibs[i - 1] + fibs[i - 2])
  return fibs

def main():
 """Prints the Fibonacci sequence up to 10."""

 print(fibonacci(10))

if __name__ == "__main__":
 main()

Use code with caution. Learn more

This code works by first defining a function called fibonacci(). This function takes a number as input and returns a list of the Fibonacci numbers up to that number. The function works by first checking if the input number is 0 or 1. If it is, the function simply returns a list containing those two numbers. Otherwise, the function creates a list of two elements, 0 and 1. Then, the function enters a loop that iterates from 2 to the input number + 1. In each iteration of the loop, the function adds the previous two elements in the list together and appends the result to the list. Finally, the function returns the list.

The code also defines a function called main(). This function simply prints the Fibonacci sequence up to 10. The function calls the fibonacci() function with 10 as input and then prints the result.

To test the code, we can run it and see if it outputs the correct Fibonacci sequence. For example, if we run the code, it should output the following:

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

This is the correct Fibonacci sequence up to 10, so we can be confident that the code works correctly.

I hope this explanation is helpful. Please let me know if you have any other questions.

HuggingChat with Llama 2

Sure, here's a Python program that calculates the Fibonacci sequence up to a given number, along with an explanation of the code and some unit tests:

def fibonacci(n):
   """
   Calculates the nth Fibonacci number.

   The Fibonacci sequence starts with 0 and 1, and each subsequent number
   is the sum of the previous two numbers. This function uses recursion to
   calculate the nth Fibonacci number.

   Args:
       n (int): The number of Fibonacci numbers to generate.

   Returns:
       int: The nth Fibonacci number.
   """
   if n <= 1:
       return n
   else:
       return fibonacci(n-1) + fibonacci(n-2)

# Unit tests
def test_fibonacci():
   assert fibonacci(0) == 0
   assert fibonacci(1) == 1
   assert fibonacci(2) == 1
   assert fibonacci(3) == 2
   assert fibonacci(4) == 3
   assert fibonacci(5) == 5
   assert fibonacci(6) == 8
   assert fibonacci(7) == 13
   assert fibonacci(8) == 21
   assert fibonacci(9) == 34
   assert fibonacci(10) == 55

# Example usage
print(fibonacci(5)) # Should print 5
print(fibonacci(8)) # Should print 21
print(fibonacci(10)) # Should print 55

Let's go through the code step by step:

• def fibonacci(n):: This line defines a function named fibonacci that takes an integer argument n.
• if n <= 1:: This line checks whether n is less than or equal to 1. If it is, the function returns n, because the first two Fibonacci numbers are 0 and 1.
• else:: This line indicates that if n is greater than 1, the function will use recursion to calculate the next Fibonacci number.
• return fibonacci(n-1) + fibonacci(n-2): This line calls the fibonacci function recursively with arguments n-1 and n-2. It then adds the results together to get the nth Fibonacci number.

Now let's talk about the unit tests. Unit tests are a great way to ensure that your code works correctly and catch any bugs before they cause problems. In this case, we've written seven unit tests to check that the fibonacci function produces the correct output for various inputs.

The first test, assert fibonacci(0) == 0, checks that the function returns 0 when given input 0. The second test, assert fibonacci(1) == 1, checks that the function returns 1 when given input 1. These two tests cover the base cases where the input is less than or equal to 1.

The remaining five tests check the recursive case where the input is greater than 1. They assert that the function returns the correct values for inputs 2, 3, 4, 5, and 6.

Finally, we've included some example usage of the function to demonstrate how it can be called with different inputs.

Overall, this code provides a simple implementation of the Fibonacci sequence using recursion, along with unit tests to verify its correctness.

The three answers are proper Python code but only GPT-4 implemented what I ask "calculate the Fibonacci sequence up to a given number" and not "a given number in the Fibonacci sequence"!

My ranking is thus this one:

1. ChatGPT with GPT-4: Only right answer and good explanations
2. HuggingChat with Llama 2: Wrong answer but good explanations
3. Bard with PaLM 2: Wrong answer

Conclusion: what is the best LLM?

Here is a summary of my tests.

In the realm of AI chatbots and language models, our exploration leads us to a nuanced yet crucial insight: there is no one-size-fits-all solution. The "best" tool largely depends on your specific needs and usage scenario. We observed considerable variance in how these AI solutions fared across different categories: explaining complex concepts, providing detailed instructions, offering emotional support, creating unique content, and coding. The ranking, as we found, can also be subjective and heavily influenced by individual preferences.

From my personal perspective, each solution demonstrated its unique strengths. ChatGPT consistently provided concise, high-quality responses. Bard, with its internet connectivity, excelled in real-time investigations. Llama 2 impressed with its expressiveness and nuanced tonality.

While this article may not give a definitive answer to which AI solution reigns supreme, it sheds light on their distinct capabilities within diverse contexts. So, next time you wonder which tool to use, consider what you want from it, and then let the appropriate AI assistant take the lead!