Practical Functional Programming in JavaScript - Techniques for Composing Data

Welcome back to my series on practical functional programming in JavaScript. Today we'll go over techniques for composing data, that is best practices that make life easy when working with structured data inside and in between functions. Composing data has to do with the shape and structure of data, and is about as fundamental as transformation when it comes to functional programming in JavaScript. If all transformations are A => B, composing data deals with how exactly A becomes B when both A and B are structured data. From Geeks

Structured data is the data which conforms to a data model, has a well defined structure, follows a consistent order and can be easily accessed and used by a person or a computer program.

Structured data could represent anything from a user profile to a list of books to transactions in a bank account. If you've ever worked with database records, you've worked with structured data.

There's a ton of ways to go about composing data since the territory is still relatively undeveloped. Good data composition means the difference between easy to read/work with code and hard to maintain/annoying code. Let's visualize this by running through a structured data transformation. Here is some structured user data

const users = [
  {
    _id: '1',
    name: 'George Curious',
    birthday: '1988-03-08',
    location: {
      lat: 34.0522,
      lon: -118.2437,
    },
  },
  {
    _id: '2',
    name: 'Jane Doe',
    birthday: '1985-05-25',
    location: {
      lat: 25.2048,
      lon: 55.2708,
    },
  },
  {
    _id: '3',
    name: 'John Smith',
    birthday: '1979-01-10',
    location: {
      lat: 37.7749,
      lon: -122.4194,
    },
  },
]

Say we needed to turn this user data into data to display, for instance, on an admin panel. These are the requirements

• Only display the first name
• Show the age instead of the birthday
• Show the city name instead of the location coordinates

The final output should look something like this.

const displayUsers = [
  {
    _id: '1',
    firstName: 'George',
    age: 32,
    city: 'Los Angeles',
  },
  {
    _id: '2',
    firstName: 'Jane',
    age: 35,
    city: 'Trade Center Second',
  },
  {
    _id: '3',
    firstName: 'John',
    age: 41,
    city: 'San Francisco',
  },
]

At a high level, users is structured as an array of user objects. Since displayUsers is also an array of user objects, this is a good case for the map function. From MDN docs,

The map() method creates a new array populated with the results of calling a provided function on every element in the calling array.

Let's try to solve the problem in one fell swoop without composing any data beyond the top level mapping.

Promise.all(users.map(async user => ({
  _id: user._id,
  firstName: user.name.split(' ')[0],
  age: (Date.now() - new Date(user.birthday).getTime()) / 365 / 24 / 60 / 60 / 1000,
  city: await fetch(
    `https://geocode.xyz/${user.location.lat},${user.location.lon}?json=1`,
  ).then(res => res.json()).then(({ city }) => city),
}))).then(console.log) /* [
  { _id: '1', firstName: 'George', age: 32, city: 'Los Angeles' },
  { _id: '2', firstName: 'Jane', age: 35, city: 'Trade Center Second' },
  { _id: '3', firstName: 'John', age: 41, city: 'San Francisco' },
] */

This works, but it's a bit messy. It may benefit us and future readers of our code to split up some functionality where it makes sense. Here is a refactor of some of the above into smaller functions.

// user {
//   name: string,
// } => firstName string
const getFirstName = ({ name }) => name.split(' ')[0]

// ms number => years number
const msToYears = ms => Math.floor(ms / 365 / 24 / 60 / 60 / 1000)

// user {
//   birthday: string,
// } => age number
const getAge = ({ birthday }) => msToYears(
  Date.now() - new Date(birthday).getTime(),
)

// user {
//   location: { lat: number, lon: number },
// } => Promise { city string }
const getCityName = ({ location: { lat, lon } }) => fetch(
  `https://geocode.xyz/${lat},${lon}?json=1`,
).then(res => res.json()).then(({ city }) => city)

These functions use destructuring assignment to cleanly grab variables from object properties. Here we see the beginnings of composing data by virtue of breaking down our problem into smaller problems. When you break things down into smaller problems (smaller functions), you need to specify more inputs and ouputs. You thereby compose more data as a consequence of writing clearer code. It's clear from the documentation that getFirstName, getAge, and getCityName expect a user object as input. getAge is further broken down for a conversion from milliseconds to years, msToYears.

• getFirstName - takes a user with a name and returns just the first word of the name for firstName
• getAge - takes a user with a birthday e.g. 1992-02-22 and returns the corresponding age in years
• getCityName - takes a user with a location object { lat, lon } and returns the closest city name as a Promise.

Quick aside, what is a Promise? From MDN docs

The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value.

I won't go too much more into Promises here. Basically, if the return value is not here yet, you get a Promise for it. In getCityName, we are making a request to an external API via fetch and getting a Promise because sending a request and waiting for its response is an asynchronous operation. The value for the city name would take some time to get back to us.

Putting it all together, here is one way to perform the full transformation. Thanks to our good data composition, we can now clearly see the new fields firstName, age, and city being computed from the user object.

Promise.all(users.map(async user => ({
  _id: user._id,
  firstName: getFirstName(user),
  age: getAge(user),
  city: await getCityName(user),
}))).then(console.log) /* [
  { _id: '1', firstName: 'George', age: 32, city: 'Los Angeles' },
  { _id: '2', firstName: 'Jane', age: 35, city: 'Trade Center Second' },
  { _id: '3', firstName: 'John', age: 41, city: 'San Francisco' },
] */

This code is pretty good, but it could be better. There is some boilerplate Promise code, and I'm not the biggest fan of the way we're expressing the async user => ({...}) transformation. As far as vanilla JavaScript goes, this code is great, however, improvements could be made with library functions. In particular, we can improve this example by using all and map from my asynchronous functional programming library, rubico. And no, I don't believe we could improve this example using another library.

• map is a function pretty commonly implemented by asynchronous libraries; for example, you can find variations of map in the Bluebird and async libraries. map takes a function and applies it to each element of the input data, returning the results of the applications. If any executions are Promises, map returns a Promise of the final collection.
• You won't find all anywhere else but rubico, though it was inspired in part by parallel execution functions like async.parallel and Promise.all. all is a bit like Promise.all, but instead of Promises, it takes an array or object of functions that could potentially return Promises and evaluates each function with the input. If any evaluations are Promises, all waits for those Promises and returns a Promise of the final value.

We can express the previous transformation with functions all and map like this

// users [{
//   _id: string,
//   name: string,
//   birthday: string,
//   location: { lat: number, lon: number },
// }] => displayUsers [{
//   _id: string,
//   firstName: string,
//   age: number,
//   city: string,
// }]
map(users, all({
  _id: user => user._id,
  firstName: getFirstName,
  age: getAge,
  city: getCityName, // all and map will handle the Promise resolution
})).then(console.log) /* [
  { _id: '1', firstName: 'George', age: 32, city: 'Los Angeles' },
  { _id: '2', firstName: 'Jane', age: 35, city: 'Trade Center Second' },
  { _id: '3', firstName: 'John', age: 41, city: 'San Francisco' },
] */

No more Promise boilerplate, and we've condensed the transformation. I'd say this is about as minimal as you can get. Here, we are simultaneously specifying the output array of objects [{ _id, firstname, age, city }] and the ways we compute those values from the user object: getFirstName, getAge, and getCityName. We've also come full circle; we are now declaratively composing an array of user objects into an array of display user objects. Larger compositions are easy when you break them down into small, sensible compositions.

Of course, we've only scratched the surface. Again, there are a lot of directions your code can take when it comes to composing data. The absolute best way to compose data will come from your own experience composing data in your own code - I can only speak to my own pitfalls. With that, I'll leave you today with a rule of thumb.

• If you need to get an object or array with new fields from an existing object or an array, use all.

Thanks for reading! You can find the rest of the articles in this series on rubico's awesome resources.