DEV Community

Async/Await and the forEach Pit of Despair

Burke Holland on September 13, 2018

The JavaScript VM is like a 4 year-old on a sugar high; screaming through your code with no pants on and just executing everything at all once whil...

Read full post

simo • Sep 14 '18 • Edited

The difference between forEach and for of in your example is that with forEach you are executing all HTTP requests in parallel, and with for of sequentially.

So it's not about 'fixing' things but instead understanding them.

Hope that helps.

Thomas van Latum • Sep 14 '18 • Edited

By doing the for loop instead of the ForEach you just killed the whole purpose of doing the async code.
supposed axios(${baseURL}/users/${post.userId}) takes a couple of seconds to run...

Listen to your Toddler, he just wants to move fast and execute stuff as fast as possible!

You should just put the result in an array and sort the order out in the end.

Douglas Gubert • Sep 14 '18

Hey Burke, nice one!

I have ran into a similar problem when I was making some scripts to populate a database with some records. I had this "async loop" and wanted to wait for it to finish to show some messages and do some cleanup, but obviously the clean up was being executed before all the asynchronous tasks I've started in the .forEach had finished :(

My solution at the time was to take out the .forEach and make it something like this

await Promise.all(data.map(async () => { .... } ))

But your solution is ways more elegant!

Vincent Brouwers • Sep 14 '18

I've solved this problem before by using Array#reduce to basically build a big Promise chain.

It's something like

await posts.data.reduce((prev, curr) => prev.then(async post => {
    let user = await axios(`${baseURL}/users/${post.userId}`);
    console.log(user.data.name);
}), Promise.resolve())

If I only need the result to ordered and don't care about the actual execution order, I use Array#map with Promise#all

const result = Promise.all(posts.data.map(async post => await axios(`${baseURL}/users/${post.userId}`)))

(in this example both the async and await can be left out since only the plain promises get used, but it should get the thought across)

Dipun Mistry • Sep 15 '18

Everything you describe from a code point of view is correct, but as a developer, when you write code, you should know what you are writing. Let's talk about await async first:

async function main() {
    await somethingAsync();
    return 20;
}

function main2() {
    return somethingAsync().then(_ => 20);
}

The two methods above are identical. This is what is happening behind the scenes.

now a quick primer on forEach:

myArray.forEach(function(val) {
  return val + 20;
});

most people know that returning something here is pointless, forEach will run your function for every item in the array, but it won't do anything with what you return.

Let's go back to await async:

async function main() {
  await doSomethingAsync();
}

Although this doesn't look like it returns anything, it actually translates to this:

function main() {
  return doSomethingAsync();
}

This is why you need to await on async functions and just like in any language, the async/await paradigm bleeds out and suddenly everything is asynchronous. If you don't await on that function when you call it, you risk running your next operation before the first operation finished.

In some cases this is fine, sometimes you want your application to go off and do a bunch of things at once, eg: when a page loads in the browser, you want the browser to go fetch your images, js and css as quickly as possible, making use of all it's resources; you don't want it to fetch them one at a time.

Now remember, forEach does nothing with whatever you return in your function, but async functions need to be awaited if you want things to go in sequencial order.

Looking at your example, you might actually be missing a trick: just because the console logs need to run sequentially, doesn't mean you should force your http calls to happen sequentially:

const p_users = data.map((post, index) => axios(post, index));
const users = await Promise.all(p_users); // Promise.all maintains the order of values, and waits for all promises to resolve before continuing.
users.forEach(user => console.log(user));

The above call will allow you to make all your http calls at the same time instead of one at a time but allows you to log your messages in the correct order. Overall, same result, just faster.

Everything you saw happen, happened by design; this may be an insightful article for the next developer, but let's educate and empower without scaring and blaming.

Jon 🇪🇺 • Sep 16 '18 • Edited

await Promise.all(data.map(async () => { .... } )) proves to be more performant in my case - because it runs them all asynchronously and then resolves the list in the correct order as opposed to the for of loop which awaits on each iteration of the loop - making it slower because things don't run in parallel ...I think

Frederik 👨‍💻➡️🌐 Creemers • Sep 16 '18

You can use backticks "``" around in-line source code to improve the formatting. Just a heads-up. Also since this seems to be your first comment on DEV.to, welcome!

Mihail Malo • Sep 14 '18 • Edited

You need to be aware that you not only changed the order of the output, but also limited concurrent requests from "all at once" to "only ONE at a time" which is not what you'd often want.
If you wanted to log them asynchronously, but in the returned order, you only had to make a small change:

ConsoleLogHTML.connect(document.getElementById('log'));

const baseURL = 'https://jsonplaceholder.typicode.com';

async function main() {
  // get all posts
  const posts = await axios(`${baseURL}/posts`);
  const names = posts.data.map(async (post, index) => {
    // get the user who posted this post
    const user = await axios(`${baseURL}/users/${post.userId}`);
    return `${index}: ${user.data.name}`;
  });
  for (const name of names) {
    console.log(await name)
  }
}

main();

This way, all requests are started at the beginning, and then we go through the list of promises in order, effectively sleeping on every unfinished request.
Usually, in a situation like that I won't splurge for an inner async function, so it would actually end up looking like this:

ConsoleLogHTML.connect(document.getElementById('log'))

const baseURL = 'https://jsonplaceholder.typicode.com'

async function main() {
  // get all posts
  const posts = await axios(`${baseURL}/posts`)
  // map user ids to user name promises
  const names = posts.data.map((post, index) =>
    axios(`${baseURL}/users/${post.userId}`).then(
      user => `${index}: ${user.data.name}`
    )
  )
  for (const name of names) {
    console.log(await name)
  }
}

main()

No brakes on the ajax train

If you truly wanted to display them asynchronously, and as soon as possible, you'd reserve space for each request's output, like this:

All at once (feat. `Promise.all`)

Finally, if you only care about all the data together, and don't need it to get in front of the user's eyes as soon as humanly possible, just use Promise.all:

ConsoleLogHTML.connect(document.getElementById('log'))

const baseURL = 'https://jsonplaceholder.typicode.com'

async function main() {
  // get all posts
  let posts = await axios(`${baseURL}/posts`)
  const users = await Promise.all(
    posts.data.map(post => axios(`${baseURL}/users/${post.userId}`))
  )
  for (const user of users) {
    console.log(`${users.indexOf(user)}: ${user.data.name}`)
  }
}

main()

iurquiza • Sep 14 '18 • Edited

'forEach' and 'map' are declarative functions used to traverse/manipulate all items in an array. You are not supposed to assume how they work internally, including how they access the items. In many cases/languages they are use to parallelize execution, which basically throw order of completion out of the window. If your task depends on the order of completion use a procedural loop. forEach's internal implementation can use a typical loop but it could also be implemented using a different traversal method.

Adebiyi Adedotun • Sep 15 '18

The truth I learnt from experiment is in two parts:

According to the MDN docs,

forEach() method executes a provided function once for each array element.
Which doesn't mean sequentially (but in here, in parallel), but we are already used to executing things synchronously most of the times we never take async into consideration. So, practically, async has the right to change the order of execution, whatever promise gets the baton goes first...
I also figured something with functions that require callbacks... again the forEach() MDN says: forEach() method executes a provided function once for each array element. "Once for each array element" And based on my experience with this, what we should really aim to iterate and perform an action upon is the array(posts.data) item, in this case "post" not "user" because that breaks the purpose of the forEach() method.