DEV Community

Cover image for Never write tests or docs again! - Amalgam could change everything.
Adam Crockett πŸŒ€ for Passenger Code Alignment Assistant

Posted on • Updated on

Never write tests or docs again! - Amalgam could change everything.

Amalgam is my little experimental tool written in rust and my attempt to consider the entire software development lifecycle and question it, can we do better, can we disrupt the way we develop, (yeah, I know, I hate that phrase too) but its true, and here's how.

Your code should be an echo of your tests and docs, but it often seems to be that one of the 3 is out of sync or wrong, you and your team chase the tails trying to keep it all together but like it or not, tests can be wrong, docs can be wrong and code can somehow work but still be wrong.

How can code be wrong? πŸ€”

It can work, but it can be wrong, in the way that the party who asked for the code via a ticket for example, was misunderstood - im no 10X dev im not super human, I do misunderstand tickets, well wrote or otherwise, so I wanted to use AI to solve this. Removing the horrific pain point, realising 2 weeks later that you have wrote the wrong solution. 🀨😭

I know of Cucumber (I may say Gherkin, I mean that), in essence its formal syntax and language for generating tests from acceptance criteria - you know, that thing that makes what you’re writing actually the thing that was needed.

Scenario: Breaker guesses a word
  Given the Maker has chosen a word
  When the Breaker makes a guess
  Then the Maker is asked to score
Enter fullscreen mode Exit fullscreen mode

Because ticket writing is hard.

It is also a machine readable language and in my opinion, it is something with a barrier to entry, you need to learn but you have to commit to it to understand it let alone write it, I feel that very few teams have such time to learn it and manually write it, so enter OpenAI 🦾.

What can Amalgam and OpenAI AI do for us? πŸ€–πŸͺ„

cargo run -- docs/sample/src/coo-cart.js
Enter fullscreen mode Exit fullscreen mode

Input:

function addToCart(productId, productName, productPrice) {
  let productInCart = cart.find((product) => product.id === productId);

  if (productInCart) {
    productInCart.quantity += 1;
  } else {
    cart.push({
      id: productId,
      name: productName,
      price: productPrice,
      quantity: 1,
    });
  }

  renderCart();
}
Enter fullscreen mode Exit fullscreen mode

HTTP response:

Scenario: Add a product to the cart
    Given an empty cart
    When I add a product to the cart with id "productId", name "productName", and price "productPrice"
    Then the cart should contain the product with id "productId", name "productName", price "productPrice", and quantity 1
    And the cart should be rendered with the product details and total price
Enter fullscreen mode Exit fullscreen mode

But Adam! This isn’t how we use Cucumber πŸ₯’!! No it isn’t, it’s the reverse engineering of the traditional model, take the code and turn it into a serialised format on change. We are creating a feedback loop potentially generating and updating documentation too and creating the arse-backwards Cucumber then we can do these crazy things:

(Actually every function in this file becomes a source to generate and run tests for one, and docs for two.

But here is where it gets really saucy and new, we pass it the html page, this FAKE Jira designed to represent some source of truth

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Jira Ticket: WPTCS-001</title>
    <style>
      body {
        font-family: Arial, sans-serif;
        margin: 20px;
        background-color: #f4f5f7;
      }

      .ticket {
        background-color: #fff;
        padding: 20px;
        border-radius: 5px;
        box-shadow: 0px 0px 10px rgba(0, 0, 0, 0.1);
      }

      .header,
      .content {
        margin-bottom: 20px;
      }

      h1 {
        color: #0079bf;
        margin-bottom: 10px;
      }

      h2 {
        margin-top: 20px;
        color: #5e6c84;
      }
    </style>
  </head>

  <body>
    <div class="ticket">
      <div class="header">
        <h1>Jira Ticket: WPTCS-001</h1>
        <p>
          <strong>Epic:</strong> Nature-Themed Store Experience |
          <strong>Sprint:</strong> Sprint 5 | <strong>Priority:</strong> High
        </p>
      </div>

      <div class="content">
        <h2>Summary:</h2>
        <p>
          Develop a wood pigeon-themed cart system enhancing user experience in
          nature-themed stores.
        </p>

        <h2>Description:</h2>
        <p>
          Our objective is to introduce a cart system inspired by the aesthetics
          of a wood pigeon. The cart should be visually appealing,
          user-friendly, and resonate with our brand's commitment to nature.
        </p>

        <h2>Acceptance Criteria:</h2>
        <ul>
          <li>
            Feathered Design Aesthetics: Carts must have blue-grey and pink
            hues, handles with black striping, wood pigeon silhouette on the
            front, and signature white wing markings on the side.
          </li>
          <li>
            Cozy Nest Cart Bay: Carts stored in a "Cozy Nest Bay", allowing
            efficient stacking, utilizing recycled materials, with soft lighting
            for nighttime.
          </li>
          <li>
            Interactive Pigeon Sounds: Sensors activate cooing sounds based on
            cart interactions.
          </li>
          <li>
            QR Code Storytelling: Carts feature a QR code linking to an
            informative wood pigeon page, with engagement-based discounts.
          </li>
        </ul>

        <h2>Dependencies:</h2>
        <p>
          Collaboration with design and multimedia team for QR content.
          Consultation with hardware vendors for sound integration.
        </p>
      </div>
    </div>
  </body>
</html>
Enter fullscreen mode Exit fullscreen mode

OpenAI Will be able to figure out how closely matched the code is to the criteria giving us a score by comparing it to the Gherkin.

β€œDoes this ticket match our code?”

Outcomes, would be some sort of correctness report, run-able test for the language, documentation and guidance to improve correctness.

Difficulties

  • one file doesn’t describe a feature usually so how to collate that?
  • The length of the post limits the file sizes we can POST
  • Are you comfortable to POST your source?
  • Are you comfortable to POST your Jira?
  • What do you see are the problems?

Anyway what do you think?

Top comments (3)

Collapse
 
skryking profile image
Jason Ormes

Have you tried taking the gerkin code that you generated, making a change to it and then feeding the original function and new requirements back into the llm to see if it can alter the function to meet the new requirements? Sort of a process that could go either direction.

Collapse
 
adam_cyclones profile image
Adam Crockett πŸŒ€

I could certainly setup a watch so that when the Geirkin gets edited it tries to amend the original function, a glorified prompt, I have no doubt this would work! Business stakeholders could then code under supervision.

Collapse
 
adam_cyclones profile image
Adam Crockett πŸŒ€ • Edited

If you like the idea, here is the response to this Post as fed to ChatGPT, seemed to really like it.

The Chat