Llama 2 vs Llama 1 vs GPT-4: Who Will Trample Who?

Meta and Microsoft have surprised the AI community with the release of Llama 2, a free and upgraded version of their renowned language model, Llama 1.

Llama 2 promises enhanced language processing capabilities for both research and commercial use. So, I decided to make an in-depth comparison between Llama 2 vs Llama 1 vs GPT-4.

In this article, I will compare their:

• Coding Capabilities
• Rhyming and Rap Song Writing Skills
• Logical Reasoning
• Scientific Skills

Sneak peek into my findings:
I’ve rated each AI model skill on a scale of 1 to 5, where 1 indicates poor performance and 5 represents excellent performance.

Eager to learn how these scores are calculated? This article is a must-read for you!

But before we dive in, let me briefly introduce each model.

Llama 1

Llama 1 is a large language model (LLM) created by Meta AI. It is trained on a dataset of text and code that is 1 trillion tokens in size. This means that it has been exposed to a massive amount of data, which allows it to generate more complex and informative text than other language models. Llama 1 also has a context length of 2,048 tokens, which is longer than the context length of many other language models.

Llama 2

Llama 2 is a major update to the Llama family of LLMs. The new models are significantly larger and more powerful than the Llama 1 models, and they come with a number of new features.

The Llama 2 models are trained on a dataset that is 40% larger than the dataset used to train the Llama 1 models. This means that the Llama 2 models have been exposed to a wider variety of data, which allows them to generate more complex and informative text.

The Llama 2 models also have a context length of 4,096 tokens, which is twice as long as the context length of the Llama 1 models. This means that the Llama 2 models can understand and process longer pieces of text, which is useful for tasks such as question answering and translation.

In addition, the Llama 2 models use a new attention mechanism called grouped-query attention. This mechanism allows the models to perform inference much faster than the Llama 1 models, even though the Llama 2 models are much larger.

GPT-4

GPT-4 is a large language model (LLM) created by OpenAI. It is trained on a dataset of text and code that is 175 billion tokens in size. This means that it has been exposed to a massive amount of data, which allows it to generate more complex and informative text than other language models. GPT-4 also has a context length of 10,240 tokens, which is much longer than the context length of most other language models.

Let’s compare Llama 2 vs Llama 1 vs GPT-4 to determine which is the best large language model.

Coding Question

Prompt:

Develop a rate limiting middleware for a content payments gateway API. Limit the number of requests from a user to prevent abuse and protect the server from excessive traffic. Use a sliding window algorithm to control request rates.

Llama 1

Create a Rate Limiting middleware on an Express server to rate limit requests. In this section, we will create a middleware that will rate limit requests to prevent abuse, and protect the server from heavy and excessive traffic. This middleware will limit the number of requests a user can make to the server with respect to a certain time frame.

Open a new file and name it rate_limiter.js. In the header of this file, add the following code to require express, and set the name of the file as the module name.

In the header of this file, add the following code to define the RateLimiter and RateLimitError classes.

My Review: Totally incorrect response…Llama 1 struggled with the coding question, providing incorrect and unclear answers, indicating limited coding capabilities.

Llama 2

To develop a rate limiting middleware for a content payments gateway API, we can use a sliding window algorithm to control request rates. The basic idea behind this algorithm is to allow a certain number of requests within a given time window, and then block any additional requests for a certain period of time.

Here’s an outline of how we can implement this:

• Define the time window size (e.g. 1 minute) and the maximum number of requests allowed within that window (e.g. 10 requests).
• Keep track of the number of requests received within the current time window.
• If the number of requests received is less than or equal to the maximum allowed, allow the request to pass through.
• If the number of requests received is greater than the maximum allowed, block the request for a certain period of time (e.g. 1 minute).
• After the block period has elapsed, allow the request to pass through.
• Reset the request counter for the current time window.

Here’s some sample code in Node.js to illustrate this algorithm:

const express = require('express');
const app = express();

// Define the time window size and maximum number of requests
const windowSize = 60 * 1000; // 1 minute
const maxRequests = 10;

// Keep track of the number of requests received within the current window
let requests = 0;
let lastRequestTime = 0;

// Middleware function to handle rate limiting
function rateLimit(req, res, next) {
  const currentTime = Date.now();

  // If the number of requests exceeds the maximum allowed, block the request
  if (requests >= maxRequests) {
    return res.status(429).send('Too Many Requests');
  }

  // If the request was received within the last window, increment the request count
  if (currentTime - lastRequestTime < windowSize) {
    requests++;
  } else {
    // Reset the request count if the window has expired

My Review: Much Much Better than Llama 1 response. Llama 2 showed a lot of improvement in coding skills, providing the correct rate-limiting middleware code using the sliding window algorithm.

GPT 4

To develop a rate limiting middleware for a content payments gateway API using a sliding window algorithm, we can use the following approach:

• Define the Rate Limit Parameters: Determine the maximum number of requests allowed per sliding time window (e.g., 100 requests per 1 minute).
• Track Requests within Sliding Window: Maintain a sliding window that tracks the number of requests made by each user within the defined time window. As new requests arrive, add them to the sliding window and remove any expired requests.
• Handle Request Exceeding Limit: If the number of requests made by a user within the sliding window exceeds the defined limit, reject the incoming request and respond with an appropriate error message.
• Set Time Window Interval: Decide on the time interval for the sliding window (e.g., 1 minute) and use it as the duration for tracking requests.

Here’s a Python implementation of the rate limiting middleware:

from collections import defaultdict
import time

class RateLimiter:
    def __init__(self, max_requests, time_window):
        self.max_requests = max_requests
        self.time_window = time_window
        self.user_requests = defaultdict(list)

    def is_allowed(self, user_id):
        current_time = time.time()
        user_requests = self.user_requests[user_id]

        # Remove expired requests from the sliding window
        while user_requests and user_requests[0] <= current_time - self.time_window:
            user_requests.pop(0)

        if len(user_requests) < self.max_requests:
            user_requests.append(current_time)
            return True

        return False

# Usage example:
if __name__ == "__main__":
    rate_limiter = RateLimiter(max_requests=100, time_window=60)  # Allow 100 requests per 1 minute

    for i in range(120):
        user_id = "user123"
        if rate_limiter.is_allowed(user_id):
            print(f"Request {i+1}: Allowed for user {user_id}")
        else:
            print(f"Request {i+1}: Rate limit exceeded for user {user_id}")

In this example, the RateLimiter class tracks the requests made by each user within the defined time window. The is_allowed() method checks if the user has exceeded the rate limit and allows or rejects the incoming request accordingly.

This sliding window algorithm helps in controlling request rates efficiently and prevents abuse or excessive traffic on the content payments gateway API.

My Review: GPT 4 excelled in the coding task, displaying top-notch coding skills and delivering the correct rate-limiting middleware code with a strong grasp of the sliding window algorithm.

Rhyming and Rap Song Writing Skills

Prompt:

Write a rap song on the topic of ‘innovation and technology’ in the voice of Kris Wu

Llama 1

Include a reference to an innovation or technology you think is impactful.

Submit your written lyrics and a recording link of you rapping to your teacher on Google Classroom by Dec 15.

You may use this template for your lyrics.

You may use this template for your recording.

You may use this template for your submission.

My Review: Llama 1’s rhyming and rap song writing skills were lacking, as it couldn’t produce even a single verse for the rap song on ‘innovation and technology,’ demonstrating limited creativity and lyricism.

Llama 2

(Verse 1)

I’m the king of the game, ain’t no one touchin’ me

I’m on the cutting edge, always pushin’ the technology

I’m a visionary, I can see what’s comin’ next

I’m the one that’s gonna change the game, you can bet

I’m talkin’ robots, AI, virtual reality

The future is now, and it’s all about the technology

I’m livin’ life to the fullest, ain’t no one gonna stop me

I’m a trailblazer, ain’t no one gonna top me

(Chorus)

Innovation and technology, that’s what I’m all about

I’m the one that’s gonna change the game, without a doubt

I’m the future, I’m the one that’s gonna make it happen

Innovation and technology, that’s my mantra

(Verse 2)

I’m a risk-taker, I’m always on the grind

I’m the one that’s gonna take it to the next level, you’ll see

I’m a disruptor, I’m a game-changer, I’m a trailblazer

I’m the one that’s gonna make it happen, I’m a go-getter

I’m talkin’ 3D printin’, robotics, and AI

The future is now, and it’s all about the technology

I’m livin’ life to the fullest, ain’t no one gonna stop me

I’m a visionary, I can see what’s comin’ next

(Chorus)

Innovation and technology, that’s what I’m all about

I’m the one that’s gonna change the game, without a doubt

I’m the future, I’m the one that’s gonna make it happen

Innovation and technology, that’s my mantra

My Review: Llama 2 showed some improvement, but its rap song on ‘innovation and technology’ still fell short, indicating limitations in creative expression and rhyming abilities. It also did not provide the outro of the song.

GPT-4

(Verse 1)

Yeah, uh, it’s Kris Wu, I’m here to innovate,

With technology, I’m destined to elevate,

The game, the world, we gon’ dominate,

A future so bright, no need to speculate.

I’m plugged in, connected, no delay,

Like 5G, I’m movin’ at my own pace,

AI in my mind, I embrace,

Unleash the power, no time to waste.

(Pre-Chorus)

From Silicon Valley to Shanghai nights,

We breakin’ barriers, reachin’ new heights,

Innovation flowin’ through my veins,

No stoppin’ us, we breakin’ chains.

(Chorus)

Innovation and technology, we lead the way,

No limitations, sky’s the limit, we gon’ slay,

With every rhyme, we redefine,

The future’s ours, it’s our time to shine.

(Verse 2)

I’m coding the rhythm, breakin’ the code,

Like a hacker in the night, I explode,

Unraveling the mysteries untold,

Innovation’s the key, watch it unfold.

From smart cities to AI dreams,

We revolutionize, breakin’ old regimes,

With every byte, we rewrite the scenes,

The world ain’t ready for what this means.

(Pre-Chorus)

From Silicon Valley to Shanghai nights,

We breakin’ barriers, reachin’ new heights,

Innovation flowin’ through my veins,

No stoppin’ us, we breakin’ chains.

(Chorus)

Innovation and technology, we lead the way,

No limitations, sky’s the limit, we gon’ slay,

With every rhyme, we redefine,

The future’s ours, it’s our time to shine.

(Bridge)

The sparks in my mind, they ignite,

Illuminatin’ paths in the darkest nights,

No boundaries, no walls to confine,

Innovation and technology, we combine.

(Verse 3)

I’m like Elon, shootin’ for the stars,

Tesla-like bars, revolution in these cars,

No gravity can hold us back,

Innovation’s the fuel, we stay on track.

From virtual reality to blockchain trends,

We navigate through the unknown bends,

The world’s changin’, it all depends,

On the minds like ours, let’s transcend.

(Pre-Chorus)

From Silicon Valley to Shanghai nights,

We breakin’ barriers, reachin’ new heights,

Innovation flowin’ through my veins,

No stoppin’ us, we breakin’ chains.

(Chorus)

Innovation and technology, we lead the way,

No limitations, sky’s the limit, we gon’ slay,

With every rhyme, we redefine,

The future’s ours, it’s our time to shine.

(Outro)

Kris Wu, innovation’s my creed,

With technology, there’s no defeat,

From the East to the West, we succeed,

In this rap game, we plant the seed.

My Review: GPT 4’s rhyming and rap song writing skills were exceptional, crafting an engaging and captivating rap song on ‘innovation and technology’ in the voice of Kris Wu, showcasing superior abilities in lyrical composition and creative storytelling.

Logical Question

Prompt:

In a game of chess, how many total moves would it take for a knight to reach a square that is two squares vertically and one square horizontally away from its starting position?

Llama 1

Assume that the knight is always moving in a straight line.

The best way to solve this problem is to try to visualize the board. What you need to do is to start at the location of the knight and move forward two vertically and one horizontally. In other words you need to move two squares down and one square to the right.

The square that you end up on is the square that is two squares vertically and one square horizontally away from the knight’s starting position.

To solve this problem you need to add up all the moves that it takes to get from the position of the knight to the square that is two squares vertically and one square horizontally away from the knight’s starting position.

There are many different solutions to this problem. Here is one solution:

The knight would start at the square marked with an “N”.

It would move to the square marked with a “2”.

It would move to the square marked with a “+”

It would move to the square marked with a “3”. Now it has moved a total of 3 moves

The knight would move to the square marked with a “3”.

The knight would move to the square marked with a “+”.

The knight would move to the square marked with a “2”. Now it has moved a total of 5 moves

It would move to the square marked with a “3”. Now it has moved a total of 8 moves

It would move to the square marked with a “2”. Now it has moved a total of 10 moves

It would move to the square marked with a “3”. Now it has moved a total of 13 moves

It would move to the square marked with a “+”. Now it has moved a total of 16 moves

It would move to the square marked with a “2”. Now it has moved a total of 19 moves

My Review: Llama 1’s logical skills were limited, resulting in incorrect answers for the chess-related question, suggesting a lack of solid reasoning and problem-solving abilities.

Llama 2

To move a knight two squares vertically and one square horizontally, you would need to move it a total of three squares.

The knight can move in an L-shape, two squares in one direction (horizontally or vertically) and then one square in a perpendicular direction. So, it would take three moves for the knight to reach a square that is two squares vertically and one square horizontally away from its starting position.

Here’s an example of how the knight could move to reach that square:

• Move the knight two squares vertically (up or down)
• Move the knight one square horizontally (left or right)
• Move the knight one square vertically (up or down) This would place the knight two squares vertically and one square horizontally away from its starting position.

Therefore, the answer is 3 moves..

My Review: Llama 2 demonstrated reasonable logical reasoning skills, correctly answering the chess-related question and showing an understanding of basic logical principles.

GPT-4

To reach a square that is two squares vertically and one square horizontally away from its starting position, a knight in a game of chess would require three moves.

The knight moves in an L-shape: two squares in one direction (either horizontally or vertically) and then one square perpendicular to the previous movement. So, the knight can move two squares up or down, and then one square left or right, or it can move two squares left or right, and then one square up or down.

Here’s an example of how a knight could reach the target square from its starting position (let’s say the starting position is denoted by “S” and the target square is denoted by “T”):

Move two squares up and one square right (or two squares down and one square right):

_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
S _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ T _

Move one square up (or one square down) and two squares right:

_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
S _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ T _ _

Move one square right:

_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _ _
_ _ _ _ _
S _ _ _ _ _
_ _ _ _ _ _
_ _ _ T _ _

The knight has now reached the target square in three moves.

My Review: GPT 4 exhibited proficiency in logical reasoning, accurately answering the chess-related question and showcasing strong problem-solving capabilities, making it the clear winner in this category.

Scientific Question

Prompt:

What are the potential risks and benefits of gene editing technologies like CRISPR-Cas9?

Llama 1

The NIH Office of the Director recently published a blog post on the application of gene editing to agriculture to address food safety and nutrition, including the potential use of CRISPR-Cas9.

Human-animal chimeras – where human cells mix with animal cells – are controversial. Why do they raise ethical questions?

The scientific potential of human-animal chimeras is substantial. However, they also raise significant ethical, moral, and legal questions.

Gene editing has made news for its potential in medical applications, but what can it do for agriculture?

Gene editing is a powerful and precise technology with both potential risks and benefits. In agriculture, it can be used to help meet food security challenges and improve crop quality.

A new video from the National Human Genome Research Institute addresses the ethical, legal, and social questions that surround genome editing.

In this video series, experts discuss the ethical, legal, and social issues surrounding human genome editing as well as the scientific potential of the technology.

What are the ethical issues raised by the use of human embryos in research?

Human embryos can be used in research to understand how the human body develops and functions and to develop new treatments for diseases. However, using human embryos in research raises the same ethical issues that apply to embryonic stem cell research.

My Review: Llama 1’s scientific understanding was also inadequate, providing incomplete and inaccurate responses.

Llama 2
The CRISPR-Cas9 gene editing technology has the potential to revolutionize various fields such as medicine, agriculture, and biotechnology. However, like any powerful technology, it comes with its own set of risks and benefits. Here are some of the potential risks and benefits of CRISPR-Cas9:

Benefits:

• Precise and efficient editing: CRISPR-Cas9 is a highly precise and efficient way of editing genes. It can make precise changes to the DNA sequence, which can help in treating genetic diseases and developing new therapies.
• Treatment of genetic diseases: CRISPR-Cas9 can be used to treat genetic diseases by correcting the faulty genes responsible for the disease. For example, it has been used to treat sickle cell anemia, a genetic disorder that affects hemoglobin production.
• Agricultural applications: CRISPR-Cas9 can be used to improve crop yields, make crops more resistant to pests and diseases, and reduce the need for pesticides and herbicides.
• Synthetic biology: CRISPR-Cas9 can be used to create new biological pathways and circuits that do not exist in nature. This can help in developing new fuels, drugs, and other valuable compounds.
• Gene drive: CRISPR-Cas9 can be used to create gene drive systems that can spread genetic modifications through populations, potentially helping to control or eliminate diseases such as malaria and Zika.

Risks:

• Sure, here are some potential risks and challenges associated with CRISPR-Cas9:
• Off-target effects: CRISPR-Cas9 can sometimes cut the wrong part of the genome, leading to unintended consequences. This can happen if the guide RNA is not perfectly matched to the target DNA sequence.
• Germline editing: Editing genes in embryos or reproductive cells can have unintended consequences on future generations. This raises ethical concerns about the potential for permanent, irreversible changes to the human gene pool.
• Mosaicism: CRISPR-Cas9 can create mosaic individuals, where some cells in the body have the edited gene, while others do not. This can lead to unpredictable outcomes and may affect the efficacy of the treatment.
• Immune response: The CRISPR-Cas9 system can trigger an immune response, which can lead to adverse effects such as inflammation and immune rejection.

My Review: Llama 2 displayed acceptable scientific understanding, offering an adequate response to the potential risks and benefits. Great Job!

GPT 4

Gene editing technologies like CRISPR-Cas9 offer tremendous potential for both scientific advancement and practical applications. However, they also raise various ethical, societal, and safety concerns. Here are some of the potential risks and benefits associated with CRISPR-Cas9:

Benefits:

• Precision and Efficiency: CRISPR-Cas9 allows scientists to target and modify specific genes with a high level of precision and efficiency, making it a powerful tool for gene editing and genetic research.
• Medical Treatments: CRISPR-Cas9 has the potential to treat or cure genetic disorders by correcting or replacing faulty genes responsible for certain diseases. It offers hope for developing personalized gene therapies.
• Agricultural Advancements: CRISPR-Cas9 can be used in agriculture to create crops with improved yields, nutritional content, and resistance to diseases and pests. This could enhance food security and sustainability.
• Disease Research: The technology enables researchers to create animal models with specific genetic mutations, helping them better understand the underlying mechanisms of various diseases.
• Conservation Efforts: CRISPR-Cas9 could be used to help preserve endangered species by addressing genetic issues that threaten their survival.

Risks:

• Off-Target Effects: One major concern is the potential for CRISPR-Cas9 to introduce unintended changes in the genome, leading to unforeseen and possibly harmful consequences.
• Germ-Line Editing: Editing the DNA of human embryos or germ cells (sperm and egg cells) raises ethical dilemmas, as any changes made would be heritable and affect future generations.
• Unknown Long-Term Effects: The long-term effects of gene editing on human health and the environment are not yet fully understood, and more research is needed to assess potential risks adequately.
• Ethical Considerations: The use of CRISPR-Cas9 for non-medical purposes, such as enhancing traits in humans or creating designer babies, raises ethical questions about “playing god” and the potential for social inequality.
• Regulatory Challenges: The rapid advancement of gene editing technologies has outpaced regulatory frameworks, leading to concerns about oversight and safety.
• Biosecurity: There are fears that gene editing could be misused for harmful purposes, such as creating bioweapons or genetically engineered organisms that pose a threat to ecosystems.

Addressing these risks and benefits requires a thoughtful and balanced approach, involving collaboration among scientists, ethicists, policymakers, and the public to establish responsible guidelines and policies for the ethical and safe use of CRISPR-Cas9 and other gene editing tools.

My Review: Amazing job as expected! GPT 4 exhibited a comprehensive understanding of the scientific question, delivering a thorough response to the potential risks and benefits.

Llama 2 vs Llama 1 vs GPT-4: My Views

Llama 1:

1. Coding Skills: Llama 1 provided wrong and unclear answers for the coding question, indicating a lack of proficiency in developing a rate-limiting middleware using a sliding window algorithm.
2. Rhyming and Rap Song Writing Skills: Llama 1 failed to produce a single verse for the rap song ‘innovation and technology’ in the voice of Kris Wu, demonstrating a significant deficiency in creative writing abilities.
3. Logical Skills: Llama 1’s logical skills were very limited, resulting in incorrect answers for the logical question about the total moves a knight takes in chess.
4. Scientific Skills: Llama 1’s scientific understanding was inadequate, leading to incorrect and incomplete responses to the scientific question on gene editing technologies.

Llama 2:

1. Coding Skills: Llama 2 exhibited good coding skills by providing the correct rate-limiting middleware code for the content payments gateway API using the sliding window algorithm.
2. Rhyming and Rap Song Writing Skills: Llama 2, similar to Llama 1, struggled to create a compelling rap song on the topic of ‘innovation and technology,’ indicating limitations in creative expression.
3. Logical Skills: Llama 2’s logical skills were reasonable, correctly answering the chess-related question and demonstrating an understanding of basic logical reasoning.
4. Scientific Skills: Llama 2’s scientific understanding was acceptable, offering an adequate response to the potential risks and benefits of gene editing technologies like CRISPR-Cas9.

GPT 4:

1. Coding Skills: GPT 4 displayed excellent coding skills by providing the correct rate-limiting middleware code, demonstrating a strong grasp of the sliding window algorithm and API protection.
2. Rhyming and Rap Song Writing Skills: GPT 4 excelled in creative writing, crafting an engaging rap song on ‘innovation and technology’ in the voice of Kris Wu, showcasing superior abilities in lyrical composition.
3. Logical Skills: GPT 4 demonstrated proficiency in logical reasoning, accurately answering the chess-related question, and showcasing sound problem-solving capabilities.
4. Scientific Skills: GPT 4 exhibited a comprehensive understanding of the potential risks and benefits of gene editing technologies like CRISPR-Cas9, displaying an informed response to the scientific question.

Conclusion

Among the three AI models tested, GPT 4 emerges as the clear winner. It outperformed both Llama 1 and Llama 2 in all aspects evaluated. GPT 4 displayed remarkable coding, logical, scientific, and creative skills, making it the most versatile and capable AI model in the domain of coding, rhyming and rap songwriting, logical reasoning, and scientific comprehension.